Thunderbird: Next Generation
Hello all,
I've been meaning to write the architecture documents only later, but
gNeandr's eagerness to start has motivated me to write at least a first
sketchy part of it already on the wiki:
https://wiki.mozilla.org/Thunderbird/NextGeneration
Particularly important and new are the sections Model and Collections. A
consistent application of it will implicitly solve a large number of
problems, including reduction of API functions, async handling, drastic
reduction of UI code, dynamic UI updates, stable extension APIs etc.
Section "async functions" could need some expansion with more concrete
details.
Ben
On 8/14/2017 7:43 PM, Ben Bucksch via Maildev wrote:
Hello all,
I've been meaning to write the architecture documents only later, but
gNeandr's eagerness to start has motivated me to write at least a
first sketchy part of it already on the wiki:
https://wiki.mozilla.org/Thunderbird/NextGeneration
Particularly important and new are the sections Model and Collections.
A consistent application of it will implicitly solve a large number of
problems, including reduction of API functions, async handling,
drastic reduction of UI code, dynamic UI updates, stable extension
APIs etc.
Section "async functions" could need some expansion with more concrete
details.
I haven't read it fully.
I dislike the "only load is async, everything else is sync" motto of
avoiding async. The truth is, synchronous interfaces really invade the
API and are really hard to undo if we ever find out it's a mistake
later. If we don't have really good API design up front, we'll end up
with a situation where some interface is sync but needs some async
feature implementation and we'll really come to regret that. (A good
example in the current codebase is nsIMsgSearchTerm--can't implement
search-in-body very well because it requires sync access). It also
requires pretty much loading anything you might want to query later in
the load function to avoid a later sync call.
Another comment about localization--how does it interact with L20N?
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
On 8/16/2017 4:15 PM, Joshua Cranmer 🐧 via Maildev wrote:
I dislike the "only load is async, everything else is sync" motto of
avoiding async.
For the db, I think that the model that we have talked about is an async
load of the equivalent of nsIMsgDBHdr, but a sync read of that object
once loaded. Isn't that what Ben is talking about, and isn't that
generally the right approach?
What does not work, again using the summary db as an example, is an
async load of nsIMsgDatabase, but then a sync read of any nsIMsgDBHdr in
that database. That forces loading large amounts of data in memory to
allow the sync read of individual items in a collection. That does not
scale to very large folders, which I would hope would be a design goal
of an improved Thunderbird. So the loading of the individual item in a
collection must be async, but the reading of properties from that item,
once loaded, should be sync.
With promises and async/await in JS, async protocols are really not that
hard to write. Can't speak to the performance of that though.
:rkent
Joshua Cranmer 🐧 via Maildev wrote on 17.08.2017 01:15:
Section "async functions" could need some expansion with more
concrete details.
I dislike the "only load is async, everything else is sync" motto of
avoiding async.
Yeah, that async section is really just a start.
The truth is, synchronous interfaces really invade the API and are
really hard to undo if we ever find out it's a mistake later.
The same is true vice-versa as well. If an API is async, all callers
(which need the result) need to be async, too.
There's no clear-cut solution for here, and I don't claim to have it.
Thanks for the concrete examples. It's easier to discuss concretely.
What does not work, again using the summary db as an example, is an
async load of nsIMsgDatabase, but then a sync read of any nsIMsgDBHdr
in that database. That forces loading large amounts of data in memory
to allow the sync read of individual items in a collection. That does
not scale to very large folders, which I would hope would be a design
goal of an improved Thunderbird. So the loading of the individual item
in a collection must be async, but the reading of properties from that
item, once loaded, should be sync.
Agreed here. And I agree with your rationale. We should assume that we
need to async load an email. Once done, we should be able to sync access
its properties.
Address book we already discussed, in the other thread.
Ben
On 8/16/2017 6:29 PM, R Kent James via Maildev wrote:
On 8/16/2017 4:15 PM, Joshua Cranmer 🐧 via Maildev wrote:
I dislike the "only load is async, everything else is sync" motto of
avoiding async.
For the db, I think that the model that we have talked about is an
async load of the equivalent of nsIMsgDBHdr, but a sync read of that
object once loaded. Isn't that what Ben is talking about, and isn't
that generally the right approach?
I'm not sure. In deference to his comments on another thread about
"loading the entire address book in memory", I assumed he meant the
database load was async and the queries thereafter were sync, which I
think is a mistake. I do agree that an async load of an nsIMsgDBHdr-like
object whose reads (but not writes) are synchronous makes sense. There
are complications around things like threads, but the basic model seems
sound.
With promises and async/await in JS, async protocols are really not
that hard to write. Can't speak to the performance of that though.
In the abstract, it seems like backend APIs should be default-async. The
question of where to draw the line is hard without concrete examples though.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
On 8/16/2017 7:07 PM, Ben Bucksch via Maildev wrote:
Joshua Cranmer 🐧 via Maildev wrote on 17.08.2017 01:15:
Section "async functions" could need some expansion with more
concrete details.
I dislike the "only load is async, everything else is sync" motto of
avoiding async.
Yeah, that async section is really just a start.
The truth is, synchronous interfaces really invade the API and are
really hard to undo if we ever find out it's a mistake later.
The same is true vice-versa as well. If an API is async, all callers
(which need the result) need to be async, too.
There's no clear-cut solution for here, and I don't claim to have it.
Thanks for the concrete examples. It's easier to discuss concretely.
I think it would be a useful endeavor to sketch out an API for the most
important components--particularly the database and the protocol
implementation interface. I started a strawman database design here
years ago: https://old.etherpad-mozilla.org/tb-new-db-api. We probably
need a reasonably-complete API so that we can see if the design can
cover all the bases well, but I don't think we need to be strongly
beholden to the actual mechanics of the design.
Some more thoughts I've had since the last email:
- Preferences. These can crop up very deep in backend implementations
even--there's a contributor who wanted a preference knob to hide the
timezone when emitting the RFC 822 Date header. - The XPCOM-like service object. There are a few things that you need
to get that don't feel like dependencies you can pass down the
chain--preferences being an example. This may mean we need a
hashtable service provider that anyone can get any registered
service from, although that may be overkill. A similar factory
system is going to be necessary for extension support. - Startup. There are some things (preferences, again!) that make sense
to async load on startup, but otherwise stay sync the entire process.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist