maildev@lists.thunderbird.net

Thunderbird email developers

View all threads

thunderbird near term focus

MM
Magnus Melin
Thu, Oct 25, 2018 12:42 PM

Below I've noted down some work packages to concretize my thoughts on
where I think we should be putting resources into in the near future. We
will make more detailed plans as we go, but as a starting point.
Comments welcome! For commenting, please quote only the relevant item
(you know, select+reply).

Thunderbird will be bringing on more staff in the near future, so these
are all items that could end up in someone's deliverables soon. Stay
tuned for an official job posting! If you already notice something you'd
be the perfect candidate for, please send a cover letter and CV to
jobs@thunderbird.net

 -Magnus


Address UI slowness
  • analyze UI slowness to and identify pain points
  • create tests to automate measuring of UI operational performance
  • perform architectural changes to support smoother operations
  • rewrites leveraging web technologies (especially moving load to
    workers where possible)
  • multi-process Thunderbird, using Frame scripts etc.

UI jank of various kinds is a top complaint from our users. We should
pinpoint the causes of these and fix them. I think having automated
tests so that we can measure progress would be important to see things
go in the right direction.

Integration improvements
  • better Gmail support
    o and get started with a global storage implementation (with
    folders being labels, like in Gmail)
    o of our users a great deal of them use Gmail, so it makes sense
    for us to improve supporting their quirks

  • better notifications (system integrated)
    o our notifications are currently less useful than they could be.
    There was earlier work to unify the implementation across
    platforms, but this was not completely finished. We can likely
    drop a lot of platform dependent implementations and unify the
    "content" production logic, and then send /that/ off to the OS
    specific notification handling.

  • improved .ics handling
    o handle inline event display better (and prettier)
    o .ics system integration (click to handle, from inside the
    application, and OS handling of .ics)
    o improvements to invites (and at the easy cases for free text),
    to see the new event details in context before taking action
    + this is an area where we can shine in privacy

    Finish rewriting nsMsgSend

  • finishing Joshua's sending rework (with ultimate goal of removing
    libmime)
    o see
    https://mail.mozilla.org/pipermail/tb-planning/2016-January/004351.html
    and bug 1211292
    o verify, complete and add tests to for converting MIME tree to an
    RFC 822 blob
    o implement building the MIME tree
    o implement support for the actual sending steps (via
    SMTP/NNTP/save-to-folder/save-to-file)

If we ever going to be get rid of libmime, this is the first step
required. Once we have the putting-together-a-messsage functionality
done, the next step would be using JSMime for body display and
ultimately remove it altogether.

Rewrite and improve mail filters
  • move filtering to async JavaScript implementation (filtering
    currently done synchronously in C++)
  • make filtering contextual (currently per server, we want it to be
    potentially global, or potentially just applying to a single folder)
  • add ability for pre-filter MIME processing, so that filtering can
    work on a message representation instead of on the raw MIME
  • investigate interaction with servers:
    o Sieve scripts through filtering - ManageSieve protocol support
    o interaction with Google filters (through gdata api)

Filters appears to be very well suited for a JavaScript rewrite. The
async nature that would be preferable is much more easily done in
JavaScript than in C++. Our filter code is very well tested too, so it's
easy to tests you didn't break it all. To address the problems of not
having filters on mobile (since we have no mobile client), make sure
filtering can sync up to the server. Assuming it's a server that have an
API for filtering - but without that we help our users. Likely to also
help remove some UI slowness.

Protocol rewrites using web technology

There are some JavaScript implementations of IMAP and SMTP at least (as
used in Gaia email), but they are missing features we need. Due to great
advances in JavaScript the last years, the libraries need modernization
to use async and Promises, and other fixes. But they could be a good
start. If we could implement an incoming mail server account type to use
the new implementation, then hopefully we can iterate from there to get
something that works well enough for general consumption.

Better, easier encryption with decent UX
  • improve the out-of-the-box encryption experience regarding encryption

  • encryption it should just work. we can't expect the end user to know
    how to renew a certificate, and it's questionable if they should be
    expected to know they have a certificate at all.

  • remove tech jargon and fix wrong/confusing messages, etc.
    o usability study attached to
    https://bugzilla.mozilla.org/show_bug.cgi?id=1243449 lists a
    bunch, but there are more :(

  • implement OTR and encrypted chats (bug 954310)

    Adopt best add-on features

  • move best of add-ons into core (but avoid gross hacks)

  • better import/export/access to own data (see ImportExportTools)

  • TNEF support - see "LookOut (fix version)"

  • best parts of Quicktext
    o but probably with a redesigned templating system (based on
    Template literals?)

  • investigate Exchange support, at least for calendar. For mail, you
    can always [try to] convince the administrator to enable imap/pop,
    but for calendar there's no other option.
    o https://github.com/ExchangeCalendar/exchangecalendar (GPL, v3)
    + see if they would be interested in re-licensing it so we can
    ship it?
    o or https://github.com/jobisoft/TbSync ? see also
    https://addons.thunderbird.net/en-US/thunderbird/addon/dav-4-tbsync/

For the most popular add-on features, it doesn't make a lot of sense to
keep them as add-ons. Most users don't know about add-ons => won't ever
find out the feature exist, and many traditional add-ons are not going
to be updated to be WebExtensions. There is of course the benefit of
having the maintenance responsibilities distributed, but it would be
preferable if the (former) add-on authors were to maintain the code
in-tree, and that way also easily get involved in other areas, as well
as in fixing actual bugs instead of working around them.

Automated testing upgrade
  • modernize thunderbird's automated testing
    o mozmill -> marionette?
    + going forwards, or port over old tests too?
    o investigate other alternatives

Mozmill is abandoned since a very long time ago, but is still our main
testing framework for UI. We should investigate moving to Marionette
like Firefox, at least for new tests.

Better support for open formats and structured data
  • CardDAV support integrated
  • WebDAV for filelink
  • vCard rewrite (starting point: rip out nsIMsgVCardService)
    o investigate if there are js libraries we could incorporate
    o vCard 3.0 and 4.0 support needed
    o CardBook (https://gitlab.com/CardBook/CardBook) code supports
    these. Possibly integrate that?
  • support sending and receiving "actions in emails" (see
    https://developers.google.com/gmail/markup/)
  • JSON-LD and RDFa Lite support (especially for copy/paste operations)
    for contacts, events and tasks.

Letting the user access their data and do cools things with it while
preserving privacy is one area where we can shine. Currently the actual
data "access" is not easy and improving it would easily make for some
really nice features. To sync contacts to 3rd parties, vCard 3.0 is
required and we only support vCard 2.0, with code dating back from
1996(!) (and as you'd imagine, looks like it too).

The complete Thunderbird package setup
  • solution for complete package (for someone wanting to use
    Thunderbird in an organization and have it "just work")
    o account setup, first run experience: polishing, set up all the
    things you need (mail, calendar, contacts, filelink, chat, and
    encryption?). DNS SRV perhaps?
  • "blessed configurations"
    o perhaps create guides and partner with some providers on how to
    set up a solution for your 100-person company?
    o docker container to deploy?
    o some way for companies to get support (for pay)
    + gate them towards sponsoring features they need too

I think there is a need for officially supported configurations.
Something the IT admin can point to when his manager asks him "what
about support"? We have a lot of pieces that can and will work after
correct setup, but it would be nice if we could point administrators to
a blessed package they could just set up and not have to worry about all
the details. Well, there are two different paths: one for
administrators, and one for end users.

Calendar improvements
  • Remove use of all calendar XPCOM components and replace them with
    simple JavaScript classes if possible.
  • Make the calendar and tasks tabs self contained, preferably using
    html only
  • Make it possible to completely defer Lightning startup, including
    the alarm service and any other background services
  • Change the Thunderbird UI Integration so that most calendar features
    are only visible once triggered, e.g. user really wants to accept a
    calendar invitation
  • Reduce or eliminate XPCOM dependencies
  • Finish ical.js and remove libical

[^^ This one mostly from Philipp]. Make calendar less reliant on Mozilla
internals. Figure out how to make ical.js performant enough to let us
move over to that completely.


Below I've noted down some work packages to concretize my thoughts on where I think we should be putting resources into in the near future. We will make more detailed plans as we go, but as a starting point. Comments welcome! For commenting, please quote only the relevant item (you know, select+reply). Thunderbird will be bringing on more staff in the near future, so these are all items that could end up in someone's deliverables soon. Stay tuned for an official job posting! If you already notice something you'd be the perfect candidate for, please send a cover letter and CV to jobs@thunderbird.net  -Magnus --- Address UI slowness * analyze UI slowness to and identify pain points * create tests to automate measuring of UI operational performance * perform architectural changes to support smoother operations * rewrites leveraging web technologies (especially moving load to workers where possible) * multi-process Thunderbird, using Frame scripts etc. UI jank of various kinds is a top complaint from our users. We should pinpoint the causes of these and fix them. I think having automated tests so that we can measure progress would be important to see things go in the right direction. Integration improvements * better Gmail support o and get started with a global storage implementation (with folders being labels, like in Gmail) o of our users a great deal of them use Gmail, so it makes sense for us to improve supporting their quirks * better notifications (system integrated) o our notifications are currently less useful than they could be. There was earlier work to unify the implementation across platforms, but this was not completely finished. We can likely drop a lot of platform dependent implementations and unify the "content" production logic, and then send /that/ off to the OS specific notification handling. * improved .ics handling o handle inline event display better (and prettier) o .ics system integration (click to handle, from inside the application, and OS handling of .ics) o improvements to invites (and at the easy cases for free text), to see the new event details in context before taking action + this is an area where we can shine in privacy Finish rewriting nsMsgSend * finishing Joshua's sending rework (with ultimate goal of removing libmime) o see https://mail.mozilla.org/pipermail/tb-planning/2016-January/004351.html and bug 1211292 o verify, complete and add tests to for converting MIME tree to an RFC 822 blob o implement building the MIME tree o implement support for the actual sending steps (via SMTP/NNTP/save-to-folder/save-to-file) If we ever going to be get rid of libmime, this is the first step required. Once we have the putting-together-a-messsage functionality done, the next step would be using JSMime for body display and ultimately remove it altogether. Rewrite and improve mail filters * move filtering to async JavaScript implementation (filtering currently done synchronously in C++) * make filtering contextual (currently per server, we want it to be potentially global, or potentially just applying to a single folder) * add ability for pre-filter MIME processing, so that filtering can work on a message representation instead of on the raw MIME * investigate interaction with servers: o Sieve scripts through filtering - ManageSieve protocol support o interaction with Google filters (through gdata api) Filters appears to be very well suited for a JavaScript rewrite. The async nature that would be preferable is much more easily done in JavaScript than in C++. Our filter code is very well tested too, so it's easy to tests you didn't break it all. To address the problems of not having filters on mobile (since we have no mobile client), make sure filtering can sync up to the server. Assuming it's a server that have an API for filtering - but without that we help our users. Likely to also help remove some UI slowness. Protocol rewrites using web technology * mail protocol rewrites into JavaScript (maybe SMTP first) and integration in Thunderbird o starting with NNTP might also be an option as it is a simple protocol and in general self contained o see http://emailjs.org/ and https://groups.google.com/forum/#!forum/emailjs There are some JavaScript implementations of IMAP and SMTP at least (as used in Gaia email), but they are missing features we need. Due to great advances in JavaScript the last years, the libraries need modernization to use async and Promises, and other fixes. But they could be a good start. If we could implement an incoming mail server account type to use the new implementation, then hopefully we can iterate from there to get something that works well enough for general consumption. Better, easier encryption with decent UX * improve the out-of-the-box encryption experience regarding encryption * encryption it should just work. we can't expect the end user to know how to renew a certificate, and it's questionable if they should be expected to know they have a certificate at all. * remove tech jargon and fix wrong/confusing messages, etc. o usability study attached to https://bugzilla.mozilla.org/show_bug.cgi?id=1243449 lists a bunch, but there are more :( * implement OTR and encrypted chats (bug 954310) Adopt best add-on features * move best of add-ons into core (but avoid gross hacks) * better import/export/access to own data (see ImportExportTools) * TNEF support - see "LookOut (fix version)" * best parts of Quicktext o but probably with a redesigned templating system (based on Template literals?) * investigate Exchange support, at least for calendar. For mail, you can always [try to] convince the administrator to enable imap/pop, but for calendar there's no other option. o https://github.com/ExchangeCalendar/exchangecalendar (GPL, v3) + see if they would be interested in re-licensing it so we can ship it? o or https://github.com/jobisoft/TbSync ? see also https://addons.thunderbird.net/en-US/thunderbird/addon/dav-4-tbsync/ For the most popular add-on features, it doesn't make a lot of sense to keep them as add-ons. Most users don't know about add-ons => won't ever find out the feature exist, and many traditional add-ons are not going to be updated to be WebExtensions. There is of course the benefit of having the maintenance responsibilities distributed, but it would be preferable if the (former) add-on authors were to maintain the code in-tree, and that way also easily get involved in other areas, as well as in fixing actual bugs instead of working around them. Automated testing upgrade * modernize thunderbird's automated testing o mozmill -> marionette? + going forwards, or port over old tests too? o investigate other alternatives Mozmill is abandoned since a very long time ago, but is still our main testing framework for UI. We should investigate moving to Marionette like Firefox, at least for new tests. Better support for open formats and structured data * CardDAV support integrated * WebDAV for filelink * vCard rewrite (starting point: rip out nsIMsgVCardService) o investigate if there are js libraries we could incorporate o vCard 3.0 and 4.0 support needed o CardBook (https://gitlab.com/CardBook/CardBook) code supports these. Possibly integrate that? * support sending and receiving "actions in emails" (see https://developers.google.com/gmail/markup/) * JSON-LD and RDFa Lite support (especially for copy/paste operations) for contacts, events and tasks. Letting the user access their data and do cools things with it while preserving privacy is one area where we can shine. Currently the actual data "access" is not easy and improving it would easily make for some really nice features. To sync contacts to 3rd parties, vCard 3.0 is required and we only support vCard 2.0, with code dating back from 1996(!) (and as you'd imagine, looks like it too). The complete Thunderbird package setup * solution for complete package (for someone wanting to use Thunderbird in an organization and have it "just work") o account setup, first run experience: polishing, set up all the things you need (mail, calendar, contacts, filelink, chat, and encryption?). DNS SRV perhaps? * "blessed configurations" o perhaps create guides and partner with some providers on how to set up a solution for your 100-person company? o docker container to deploy? o some way for companies to get support (for pay) + gate them towards sponsoring features they need too I think there is a need for officially supported configurations. Something the IT admin can point to when his manager asks him "what about support"? We have a lot of pieces that can and will work after correct setup, but it would be nice if we could point administrators to a blessed package they could just set up and not have to worry about all the details. Well, there are two different paths: one for administrators, and one for end users. Calendar improvements * Remove use of all calendar XPCOM components and replace them with simple JavaScript classes if possible. * Make the calendar and tasks tabs self contained, preferably using html only * Make it possible to completely defer Lightning startup, including the alarm service and any other background services * Change the Thunderbird UI Integration so that most calendar features are only visible once triggered, e.g. user really wants to accept a calendar invitation * Reduce or eliminate XPCOM dependencies * Finish ical.js and remove libical [^^ This one mostly from Philipp]. Make calendar less reliant on Mozilla internals. Figure out how to make ical.js performant enough to let us move over to that completely. ------------------------------------------------------------------------
JB
John Bieling
Thu, Oct 25, 2018 2:01 PM

Am 25.10.2018 um 14:42 schrieb Magnus Melin:

investigate Exchange support, at least for calendar. For mail, you can
always [try to] convince the administrator to enable imap/pop, but for
calendar there's no other option.

I have written a small "where is this going" summary for TbSync and sync
support in Thunderbird in general:
https://github.com/jobisoft/TbSync/blob/master/README.md#where-is-this-going

I am open to a better integration, but like to keep my donation revenue :-)

John

Am 25.10.2018 um 14:42 schrieb Magnus Melin: > investigate Exchange support, at least for calendar. For mail, you can > always [try to] convince the administrator to enable imap/pop, but for > calendar there's no other option. > > * https://github.com/ExchangeCalendar/exchangecalendar (GPL, v3) > o see if they would be interested in re-licensing it so we can > ship it? > * or https://github.com/jobisoft/TbSync ? see also > https://addons.thunderbird.net/en-US/thunderbird/addon/dav-4-tbsync/ > I have written a small "where is this going" summary for TbSync and sync support in Thunderbird in general: https://github.com/jobisoft/TbSync/blob/master/README.md#where-is-this-going I am open to a better integration, but like to keep my donation revenue :-) John