Infrastructure status for week ending August 20, 2017
This week:
- Figured out set up of the addons-server container both locally and
on our infrastructure.
o Still need to do dependency triage here, they use a lot of
external deployment units and I think we only need DB,
memcached, and maybe celery ones. - Research into the APIs provided by addons-server and what we need
and don't need. - Related conversations with jorgev(on the AMO team), who offered the
option of them hosting our version of addons-server temporarily.
o This would give us extra time to get opt-ins from developers,
because Mozilla can't give us the email addresses of developers
in a big import dump without opt-ins due to privacy reasons.
It's difficult to create accounts/verify add-on ownership
without email addresses. Plus, they can't give us any data for
non-open-source add-ons at all without opt-in.
o Still discussing exactly how this would work. - Updated thunderbird-website documentation
https://github.com/thundernest/thunderbird-website. - Follow up on last week's change: Web server CPU load looks much
better now http://i.imgur.com/TRJ2gR6.png, consistently staying
under 100% instead of spiking to 300%+.
o session stickiness caused this, which implies a lot of requests
are being made in the span of one session. This might be a bot
or something, should look into it. - Brief(3 min this week) web server 404s were caused by a lazy deploy
script, fixed this when monitoring presented the issue. End-users
would mostly not have experienced any issue, however, because of
Cloudflare(only uncached URLs would have 404ed). - Updated https://www.thunderbird.net/en-US/organizations/ with
Wayne's patch. - Mozilla webdev might use a version of
https://github.com/mozilla/release-notes our .yml text file
strategy for Firefox release notes instead of their current database
system.
Next week:
- More AMO.
- A complete plan for what changes need to be made for our version of
addons-server. - The usual minor bug fixes and server monitoring.
On 22/08/2017 08:49, Andrei Hajdukewycz via Maildev wrote:
Mozilla webdev might use a version of our .yml text file strategy for Firefox release notes instead of their current database system.
Hi,
as you know, I've done release notes for TB 52.3 and TB 56 beta 1/2/3 on the old system, so perhaps for the next release notes it's time to switch over to the new.
So I'd like to book a private session on how to use the new system via Skype.
Also, I'd like to know what the requisites are for Git. I know absolutely nothing about Git. Over the years I've acquired a fair working knowledge of Mercurial, hey, I'm the guy who gets to do the branches on Mozilla beta and ESR. I mainly use the command line but also TortoiseHG.
For Git, even the terminology seems to be different. In Mercurial you pull from the central repository and push changes back, on Git you seem to send pull requests, so the owner of the repository pulls in your changes. Or maybe I got that wrong.
Is there a preferred Git client? I installed "git for Windows" (https://git-for-windows.github.io/) back in 2014 before joining TB, but I can't even remember now what I used it for, if at all. There is also TortoiseGit, which I might give a go since I've had a good experience with TortoiceHG and before TortoiseSVN (in 2012! on another project).
So there you go.
Jörg.
On 2017-08-22 12:24 AM, Jörg Knobloch via Maildev wrote:
On 22/08/2017 08:49, Andrei Hajdukewycz via Maildev wrote:
Mozilla webdev might use a version of
https://github.com/mozilla/release-notes our .yml text file
strategy for Firefox release notes instead of their current database
system.
Hi,
as you know, I've done release notes for TB 52.3 and TB 56 beta 1/2/3
on the old system, so perhaps for the next release notes it's time to
switch over to the new.
Well, it'd be nice, but I'm still waiting on Mozilla webdev to give me a
plan for https://bugzilla.mozilla.org/show_bug.cgi?id=1388913. If they
don't say anything in another week or two I'll poke them.
In the meantime I've been keeping the notes on thunderbird.net up to
date with what's added on mozilla.org manually.
**
So I'd like to book a private session on how to use the new system via
Skype.
Sure. We can do that after the status meeting tomorrow even if you like.
Also, I'd like to know what the requisites are for Git. I know
absolutely nothing about Git. Over the years I've acquired a fair
working knowledge of Mercurial, hey, I'm the guy who gets to do the
branches on Mozilla beta and ESR. I mainly use the command line but
also TortoiseHG.
For Git, even the terminology seems to be different. In Mercurial you
pull from the central repository and push changes back, on Git you
seem to send pull requests, so the owner of the repository pulls in
your changes. Or maybe I got that wrong.
Is there a preferred Git client? I installed "git for Windows"
(https://git-for-windows.github.io/) back in 2014 before joining TB,
but I can't even remember now what I used it for, if at all. There is
also TortoiseGit, which I might give a go since I've had a good
experience with TortoiceHG and before TortoiseSVN (in 2012! on another
project).
Well first note to get HTML previews of your changes you'll need to have
python and be able to install the requirements using pip. I'm sure this
works totally fine on Windows but it's been many years since the last
time I tried to install Python on Windows so I'll be of limited help
with that.
I've always used the https://git-scm.com/downloads command line tool, I
haven't used it at all on Windows but I'm sure it's the same. I have
never used a git GUI(I rely on github for pretty printing of big diffs
and stuff like that) but I know TortoiseGIT is popular, as is KrakenGIT,
and I've heard good things about https://gitextensions.github.io/
I've barely used Mercurial but it's a distributed vcs so it probably has
a lot of similarities to git. The thunderbird-notes repo is very simple
and you only really need to know a few commands:
git clone https://github.com/thundernest/thunderbird-notes.git (same as
hg clone)
cd thunderbird-notes
git status (shows you what branch you're on and changed files, if any,
in the working tree)
git pull (I think this is the same as hg pull? It will pull commits in
from another repo and apply them to your tree, by default with no
arguments it pulls from the repo you cloned)
git add <path> (stages changes to be committed on the next commit
command, you can just git add . to add every change you've made in the
working tree)
git commit -m "commit message." (same as hg commit)
git checkout prod (switch the working tree to the prod branch. You can
also checkout by commit hash, or tags, etc etc)
git merge master (merges master branch into your current branch)
git push <remote> <branch> (similar to hg push, typically you'll do git
push origin master or git push origin prod, which pushes changes for the
given branch to the repo you cloned from)
There are only two branches in the repo, master and prod. If you push a
commit to master, it'll automatically update www-stage.thunderbird.net
with the changes, and same with prod and www.thunderbird.net itself. So
generally speaking you'll push commits to master first, and if you're
sure it looks good you'll checkout prod, merge master into it, and then
push prod which will trigger an update of the production site.
This whole description ignores pull requests because they're not really
useful if you're just working by yourself, but they're the ideal thing
to use if you have multiple developers and a review process. You can
fork a github repo to your own personal github account, make and push
changes there, and then use the github UI to trigger a pull request to
the original repo, at which point someone with commit rights to the main
repo can look at it and they have a button they can press to merge if
they're happy with the changes, or make line by line comments and wait
for updates, etc.
Hello Maildev Group
As I asked already [*] is there a definition of a common used platform
for the necessary TB tasks (infrastructure, development, TB:NG, etc)? Or
is there the idea of /intention for such an agreement?
@Jörg
what's yr platform? Do you use / have access to a Linux installation?
I moved from HG to GIT with our Reminderfox development (based on
Eclipse), found it a bit strange at the beginning but are happy nowadays
to have moved over to GIT.
These days I'm moving away from Eclipse to Atom, found good
addons/extensions, also for GIT which well integrated. But also here
more to learn.
So I would more than happy to find others to share experience for that
recent move (Atom/GIT/...)
Are there any concrete plans for:
So I'd like to book a private session on how to use the new system via
Skype.
??
Günter
*\ 10.08.2017 12:35 Re: [Maildev] The next phase of Thunderbird
development and TB:NG
On 22.08.2017 09:24, Jörg Knobloch via Maildev wrote:
On 22/08/2017 08:49, Andrei Hajdukewycz via Maildev wrote:
Mozilla webdev might use a version of
https://github.com/mozilla/release-notes our .yml text file
strategy for Firefox release notes instead of their current database
system.
Hi,
as you know, I've done release notes for TB 52.3 and TB 56 beta 1/2/3
on the old system, so perhaps for the next release notes it's time to
switch over to the new.
So I'd like to book a private session on how to use the new system via
Skype.
Also, I'd like to know what the requisites are for Git. I know
absolutely nothing about Git. Over the years I've acquired a fair
working knowledge of Mercurial, hey, I'm the guy who gets to do the
branches on Mozilla beta and ESR. I mainly use the command line but
also TortoiseHG.
For Git, even the terminology seems to be different. In Mercurial you
pull from the central repository and push changes back, on Git you
seem to send pull requests, so the owner of the repository pulls in
your changes. Or maybe I got that wrong.
Is there a preferred Git client? I installed "git for Windows"
(https://git-for-windows.github.io/) back in 2014 before joining TB,
but I can't even remember now what I used it for, if at all. There is
also TortoiseGit, which I might give a go since I've had a good
experience with TortoiceHG and before TortoiseSVN (in 2012! on another
project).
So there you go.
Jörg.
Maildev mailing list
Maildev@lists.thunderbird.net
http://lists.thunderbird.net/mailman/listinfo/maildev_lists.thunderbird.net
On 22/08/2017 11:06, neandr wrote:
@Jörg
what's yr platform? Do you use / have access to a Linux installation?
My reply to Andrei was meant as a private message, but since the list is
setup in a weird way, my "Reply" went to "Andrei Hajdukewycz via Maildev
maildev@lists.thunderbird.net" for some reason.
To answer the question: Windows, no access to Linux at the moment.
Jörg.
Jörg Knobloch via Maildev wrote on 22.08.2017 11:18:
my "Reply" went to "Andrei Hajdukewycz via Maildev
maildev@lists.thunderbird.net" for some reason.
Ah, that's why! I made the same mistake 2 weeks ago, sending a private
message with phone numbers to the public list :-(
Hey Kent,
-
could you please change the maildev list configuration to not put the
name of a person as description for the mailing list?
(This is highly misleading.) -
could you please consider not changing the "Reply" header, to not
break the TB Reply button? (I understand that 2. is an old controversial
debate https://nealey.github.io/papers/reply-to-still-harmful.html)
We're all email pros here.
Ben
(I sent this message to TB Council. This is just a read-only copy for
informational purposes. Please discuss on TB Council list.)
Andrei Hajdukewycz via Maildev wrote on 22.08.2017 08:49:
[AMO for TB]
o This would give us extra time to get opt-ins from developers,
because Mozilla can't give us the email addresses of
developers in a big import dump without opt-ins due to privacy
reasons. It's difficult to create accounts/verify add-on
ownership without email addresses. Plus, they can't give us
any data for non-open-source add-ons at all without opt-in.
I'm not sure that is true. We are officially MoFo, so this is the same
org. As long as you signed an NDA with MoFo on mozillias, you (as admin)
should be able to see the data.
Philipp, could you please clear this with MoFo? This appears to be a
major stumbling block for the migration.
Ben
On 22.08.2017 09:24, Jörg Knobloch via Maildev wrote:
Also, I'd like to know what the requisites are for Git. I know
absolutely nothing about Git.
For the theory behind, there's some good (small) paper:
"git for computer scientists"
http://eagain.net/articles/git-for-computer-scientists/
Turns out that git indeed is very simple - you'll just have to put aside
everything you've learned about SCMs in the past ;-)
Linus calls it not an SCM, but an directory content tracker. That's
basicly what it is: refs (branches, tags, etc) are just pointers to
specific snapshots, which may have a variable number of ancestor
snapshots (0 = initial, 1 = plain, >1 = merge).
A commit is made of:
- a tree (complete content snapshot)
- list of ancestors
- metadata (description, author, etc, etc)
The commit operation just adds another snapshot (taking the current
branch pointer as ancenstor) and move the branch pointer to it.
The merge operation creates a merged tree (of your current branch
and the other commit(s) to be merged in), commits that, but now with
multiple ancestors (adding the ones you merged in). Yes, you can merge
in an arbitrary number of commits (rarely used).
Remotes are also quite simple:
- on 'git remote update', you just get the remote's refs put into your
repo, in their own namespace. eg. 'refs/remotes/origin/foo' will be
exactly what the remote 'origin' had as as branch 'foo'. It's just a
full copy. (git doesn't want to checkout/commit in remote branches,
as they would be killed by next update anyways) - 'git pull' is a shorthand for remote update and merge (or rebase)
- 'git push' just sends a copy of your local branch to the remote
(after that, the remote branch will be exactly the same as yours) - per default, git does an sanity check that the pushed history is
an direct descendant of the previous one (IOW: you just have added
something ontop on the old history). It's just for preventing multiple
pushers to accidentially kill the other's commits - you can override
it (in some workflows you'll have to override it)
For Git, even the terminology seems to be different.
Correct. There should be some 'translations' somewhere in the net.
In Mercurial you
pull from the central repository and push changes back, on Git you seem
to send pull requests, so the owner of the repository pulls in your
changes. Or maybe I got that wrong.
No, pull requests entirely different. It's a workflow:
- you've forked from the mainline, did some commits and pushed it
somewhere in the public (eg. on your github account) - then you'll send the maintainer a pull requests, which is nothing more
then telling him that you've got something to look at. could be done
by mail, phone, smoke signals, whatever - the maintainer then pulls your repo and decides what to do, eg. accept
(merge into his mainline), some more discussions, etc. - github just has an technical implementation of that worklow which
saves you from several manual steps (eg. you can manage them via their
website, including merging etc)
We could use github's features for that, but we don't need to.
Is there a preferred Git client?
git command line ;-)
UI frontends are really a matter of taste. I prefer the command line,
together w/ 'tig' - a (console based history browser). But everybody
has to make his own choice here.
Eclipse also has some git integration (actually their own implementation
in pure java), and I've heared rumors about official MSVC+TFS
integration from MS themselves (according to some blog posts they've
migrated much of their own codebases from VSS to Git).
By the way: according to your name, you might be German-speaking ?
Maybe we could have some calls.
--mtx
--
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
On 22.08.2017 11:06, neandr via Maildev wrote:
These days I'm moving away from Eclipse to Atom, found good > addons/extensions, also for GIT which well integrated. But also here
more to learn.
I've also got atom on my try-out list. Doesn't seem to be in Debian
stable yet. Maybe you could give us some quickstart guide ?
--mtx
--
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
On 8/22/2017 5:44 AM, Ben Bucksch via Maildev wrote:
-
could you please change the maildev list configuration to not put
the name of a person as description for the mailing list?
(This is highly misleading.) -
could you please consider not changing the "Reply" header, to not
break the TB Reply button? (I understand that 2. is an old
controversial debate
https://nealey.github.io/papers/reply-to-still-harmful.html) We're
all email pros here.
The issue is DMARC, which will refuse to deliver email to domains that
have a DMARC policy in place if the From is an address on their domain,
but was not sent from one of their approved servers. The big problem
here is yahoo mail. There is no good solution for email lists which
DMARC seems to pretend do not exist, but the current policy was a
compromised setup by mailman to keep working.
I don't have a strong opinion here, but I run other email lists and the
current setup seems to work best for a variety of users, including
non-technical users. Funny that "We're all email pros here" but we don't
seem to check accurately where replies are being sent to.
But I'll change it after this message. If anyone stops getting email
after this, the only option is for you to change providers to someone
who does not publish a DMARC policy.
:rkent