On 4/4/2019 2:56 AM, Magnus Melin wrote:

To emphasize this point:

At a low-level, the most obviously reusable pieces are implementing the
parsing of commands (say, knowing how to extract low and highwater marks
from NNTP GROUP or LIST commands). But when you start trying to piece
together high-level commands, the tasks you're providing are things such
as "get the list of new messages", where the commands to use are heavily
dependent on what you want to represent with a new message. And if the
library picks wrong, it's going to be annoying to reuse it in another
product. The only high-level API I can see that is easily portable is
authentication, and even then, you need to define escape hatches to
support Kerberos or NTLM authentication [1]. Even worse, you probably
want some sort of abstraction over the socket API anyways because making
raw TCP sockets with support for STARTTLS is basically mapped
differently onto every networking stack, even if networking is sort-of
standardized on a platform.

In the end, this means there's a narrow path between "too simple to be
worth spinning off to another library" and "too tied to a particular
use-case to be worth adapting." My opinion is that it's the sort of
thing that should be worth considering as we design systems, but not
requiring it be a design feature.

[1] IMAP, once again, is a kind-of exception here. There is /a lot/ of
value to be gained merely be abstracting over details such as IMAP UID,
UTF-8/MUTF7 mailbox names, and command numbering and pipelining. But
from the vantage point of high/low-level, that's basically a statement
that IMAP's low level API is much higher than for NNTP, POP, or SMTP.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

Hey Ben C,

you're entirely correct in your analysis. thunderbird and Firefox used to be the same app and were simply separated into 2 apps, but they are architecturally identical.

i completely concur with you that the mix of C++ and JS has caused huge troubles. i would even argue that it has been the primary reason why the app code is so arcane today.

with a pure JS interface, things would be much simpler.

it was freezing both languages to the state when the bridge was invented. it was holding back JS, and also holding back the C++ side.

for the very same reason, i think adding Rust would be repeating the same mistake. we'd have the same fundamental problems of combining 2 languages. even if we could perfectly model JS and Rust in the bridge today, we'd be again limited once there are new language features.

Firefox has a good reason for Rust. they are a low level platform. they need rust.

thunderbird is an app, which is based on a platform. thunderbird has far more application surface than Firefox, and is far more vulnerable to API changes.

put another way: the Firefox application (not the engine) is also implemented in JS only. saying the Firefox is implemented partially in Rust is looking at the wrong application level.

thunderbird and the core low level browser engine have very different requirements. these result in very different solutions. following mozilla's lead here with using Rust can lead us astray, because they have different needs that don't apply to us.

i think the engineer in you sensed that, that's why you suggest to try pure JS only before attempting rust. and you're right with that.

Ben B

Am 4. April 2019 01:21:58 MESZ schrieb Ben Campbell benc@thunderbird.net:

--
Sent from my phone. Please excuse the brevity.

I've spent a lot of effort to separate the auth schemes from the protocols in current thunderbird. but the current thunderbird implementation made it very hard. it had cost a lot more time (4-10 times more than if i started from scratch) and is not as clean as it could be.

it is possible to make auth and protocol fairly independent. there's even a standard for pluggable auth schemes in mail that we support, called SASL.

a properly designed protocol library would separate authentication anyways.
if for no other reason than that the auth needs UI access and a protocol should never directly access the UI.

in fact, designing things as independent libraries forces you to design the API properly, which in turn makes the app a lot

• more stable (less avoidable API changes, better testable)
• easier to understand (which leads to Lee bugs and note contributors)
• easier to maintain
• easier to adapt to new circumstances

thunderbird could really shine in providing proven libraries that other projects can use easily. but only if the APIs are designed for reuse from the start.

Ben

Am 4. April 2019 08:56:35 MESZ schrieb Magnus Melin mkmelin+mozilla@iki.fi:

--
Sent from my phone. Please excuse the brevity.

On 07/04/19 12:22, Ben Bucksch wrote:

Always tricky in C++ - there are usually loads of low-level abstractions
to rely on (strings, timers, threads, files, sockets, platform-detection
macros, whatever) so non-trivial C++ libraries tend to have to carry a
lot of baggage with them.
Idiomatic modern C++ probably helps a here (eg less friction relying on
std::string, say, rather requiring users use some other string class),
but obviously we've got various restrictions here.

No reason not to try, of course! As you point out, there are so many
benefits, even if the end library is less stand-alone than we'd like.
It's much more likely to be easy to understand, reason about and test.

Another example candidate: mbox parsing. We've got at least two - maybe
three - separate implementations.

Ben(C).