Yes; this sounds like one of these epic challenges.
The stock browser of SFOS is pretty much abandoned to itself. It is gecko-based; gecko has been labeled as "deprecated" by Mozilla, then as "still supported", then got re-promoted to a (somewhat obscure) "we keep it until we can switch". The switch in question meaning to be the jump to servo, a rust-based web engine used in their latest quantum-based firefox. So the idea is to compile it on the x86 target to begin with.
Achieving this faces challenges.
In order to achieve this goal, we will need clang, which requires llvm and a decently recent (>=4.9.5) gcc compiler, as well a decently recent python 3 verison (>=3.5.0). On top of this, we have here:
a) the glibc in the mer toolchain (at the time of writing) is stone age (2.19)
b) the compiler in the mer toolchain (at the time of writing) is old (4.8)
c) the compilation environment, scratchbox2 (a legacy development chroot originally developed by Nokia), has aged a bit, but the taste has slightly bitterned, and might cause acidity of stomach to some newer software's.
The first two points affect mainly the correct functionality of llvm.
llvm as a compiler is a different type of beast. In that it reimplements some functionalities in it's own ways, to ensure portability across multiple platforms. For example, locks via symlinks (same as pthreads do for instance). This exact feature of llvm is the first hurdle to overcome in order to get a functional toolchain; it uncoveres a non-POSIX conformance issue in mer / scratchbox2 (on i486 architecture) related to symlinks path resolution (in the mapping logic).
scratchbox2 as a development tool is very powerful; it allows to build for multiple targets, by isolating each specialized environment, and allowing fine-grained customization for the developer; combined with easiness of switching between these. Needless to say, the flexibility of this tool is admirable. Packages can be installed in specific environments(so-called "targets"), multiple architectures are supported, targets can be exported/copied, you name it. All of this is possible thanks to an interception layer, provided via a customized pre-loader library, configured as default when entering the development MER chroot, and which takes care of mapping the active target to the "real" counterpars (in terms of files) by intercepting and managing the most well-known System calls / glibc api's (open, close, execvp and so on).
It needs to be also mentioned that nowadays (with MER merging int SFOS) the platformc chroot has been officially replaced by VM's, which ship scratchbox2 pre-installed.
The 'box2's interception layer's has limits, however, and they become visible on our journey to firefox. Take pthread, for an example; a library on it's own, but which (as an implementation) "lives" at glibc's home repository. This vicinity causes trouble for sb2, as the closeness gives way to "priviledged" relationships (in terms of api's) between pthread and glibc; now before you scream in disgust, these types of "close" relationships are very much acceptable from a glibc's or pthread's perspective (coherence of api's), if not welcomed (performance), and make sense in glibc/pthread's universe (coherence). But for the outsider's ('box2) interception layer, as an alien, this is a major problem, an interaction which is totally invisible to the SFOS development environment, and hence causes lots of things to go havoc (for example multiprocessing module of python, which internally relies on system-V semaphores).
And unfortunately, there is more glibc features which rely on many private counterparts of public api's. Up to the point where common POSIX libraries become non-functional under scratchbox2 (we already mentioned these - systemV semaphores anyone?) on some targets (i486, for instance), or exec's, which also fall into the set of functions utilized internally by glibc via a private api. Exec's are for example used by the posix_spawn api's, which in turn - you know where this is going - are used (if the spawn.h header is detected, and is the case as MER ships it) by llvm.
Nevertheless, with these issues out of the way, you should be able to reliably compile the llvm. So now to the funny part, the compilation. Did I mention MER has an ancient glibc? (e)glibc 2.19 , as used (at the time of writing) by MER, lacks certain floating point operations (sse3 & more recent), which llvm relies on. Though these are not critical per se, they might turn up iffy warnings during compilation (if not bugs at runtime), hence relative build.rs files which include them in the compilation need to be patched to exclude them.
Once done so, you should have a clean LLVM implementation running on SFOS. Enjoy!
The stock browser of SFOS is pretty much abandoned to itself. It is gecko-based; gecko has been labeled as "deprecated" by Mozilla, then as "still supported", then got re-promoted to a (somewhat obscure) "we keep it until we can switch". The switch in question meaning to be the jump to servo, a rust-based web engine used in their latest quantum-based firefox. So the idea is to compile it on the x86 target to begin with.
Achieving this faces challenges.
In order to achieve this goal, we will need clang, which requires llvm and a decently recent (>=4.9.5) gcc compiler, as well a decently recent python 3 verison (>=3.5.0). On top of this, we have here:
a) the glibc in the mer toolchain (at the time of writing) is stone age (2.19)
b) the compiler in the mer toolchain (at the time of writing) is old (4.8)
c) the compilation environment, scratchbox2 (a legacy development chroot originally developed by Nokia), has aged a bit, but the taste has slightly bitterned, and might cause acidity of stomach to some newer software's.
The first two points affect mainly the correct functionality of llvm.
llvm as a compiler is a different type of beast. In that it reimplements some functionalities in it's own ways, to ensure portability across multiple platforms. For example, locks via symlinks (same as pthreads do for instance). This exact feature of llvm is the first hurdle to overcome in order to get a functional toolchain; it uncoveres a non-POSIX conformance issue in mer / scratchbox2 (on i486 architecture) related to symlinks path resolution (in the mapping logic).
scratchbox2 as a development tool is very powerful; it allows to build for multiple targets, by isolating each specialized environment, and allowing fine-grained customization for the developer; combined with easiness of switching between these. Needless to say, the flexibility of this tool is admirable. Packages can be installed in specific environments(so-called "targets"), multiple architectures are supported, targets can be exported/copied, you name it. All of this is possible thanks to an interception layer, provided via a customized pre-loader library, configured as default when entering the development MER chroot, and which takes care of mapping the active target to the "real" counterpars (in terms of files) by intercepting and managing the most well-known System calls / glibc api's (open, close, execvp and so on).
It needs to be also mentioned that nowadays (with MER merging int SFOS) the platformc chroot has been officially replaced by VM's, which ship scratchbox2 pre-installed.
The 'box2's interception layer's has limits, however, and they become visible on our journey to firefox. Take pthread, for an example; a library on it's own, but which (as an implementation) "lives" at glibc's home repository. This vicinity causes trouble for sb2, as the closeness gives way to "priviledged" relationships (in terms of api's) between pthread and glibc; now before you scream in disgust, these types of "close" relationships are very much acceptable from a glibc's or pthread's perspective (coherence of api's), if not welcomed (performance), and make sense in glibc/pthread's universe (coherence). But for the outsider's ('box2) interception layer, as an alien, this is a major problem, an interaction which is totally invisible to the SFOS development environment, and hence causes lots of things to go havoc (for example multiprocessing module of python, which internally relies on system-V semaphores).
And unfortunately, there is more glibc features which rely on many private counterparts of public api's. Up to the point where common POSIX libraries become non-functional under scratchbox2 (we already mentioned these - systemV semaphores anyone?) on some targets (i486, for instance), or exec's, which also fall into the set of functions utilized internally by glibc via a private api. Exec's are for example used by the posix_spawn api's, which in turn - you know where this is going - are used (if the spawn.h header is detected, and is the case as MER ships it) by llvm.
Nevertheless, with these issues out of the way, you should be able to reliably compile the llvm. So now to the funny part, the compilation. Did I mention MER has an ancient glibc? (e)glibc 2.19 , as used (at the time of writing) by MER, lacks certain floating point operations (sse3 & more recent), which llvm relies on. Though these are not critical per se, they might turn up iffy warnings during compilation (if not bugs at runtime), hence relative build.rs files which include them in the compilation need to be patched to exclude them.
Once done so, you should have a clean LLVM implementation running on SFOS. Enjoy!
1 comment:
While challenges are nice at times :), there is some ease coming to your pain soon as we are working on many updates in the core platform. Here is quick summary of things:
Python got updated to version 3.7.2 in just released 3.0.2.8 release. https://together.jolla.com/question/201103/changelog-302-oulanka/#201103-python3-base
glibc is going to be updated in next release to 2.25 version and then again later when we have the remaining issues solved while doing the upgrade. Most likely next step will be 2.28. You can follow the progress at https://git.sailfishos.org/mer-core/glibc/commits/master
gcc is also getting a small update to 4.9.4 in next release, which will be stepping stone for further updates. More about the gcc progress can be seen at https://git.sailfishos.org/mer-core/gcc/commits/master
Looking forward for the new adventures,
Sage
Post a Comment