Difference between revisions of "Dependency Updates"
m (→Nightlies: format) |
(→Pull Requests at Flathub: GNOME schedule) |
||
(15 intermediate revisions by 2 users not shown) | |||
Line 3: | Line 3: | ||
'''Keep in mind that these binary distributions are for ordinary users.''' As a general rule only '''''stable release''''' dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team. | '''Keep in mind that these binary distributions are for ordinary users.''' As a general rule only '''''stable release''''' dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team. | ||
− | + | ;Developers with push privileges: unless you're able to | |
− | * test the new dependency version yourself, | + | :* test the new dependency version yourself, |
− | * the upgrade is stable, and | + | :* the upgrade is stable, and |
− | * you're certain that there won't be a negative impact on users | + | :* you're certain that there won't be a negative impact on users |
− | ''create a pull request'' on Github so that others in the core team can test the change. | + | :''create a pull request'' on Github so that others in the core team can test the change. |
+ | |||
+ | [[Dependency Sources]] has a list of URLs to check for updates. | ||
+ | [[Category:Packaging]] | ||
==Linux Flatpak== | ==Linux Flatpak== | ||
[https://github.com/gnucash/gnucash-on-flatpak gnucash-on-flatpak] | [https://github.com/gnucash/gnucash-on-flatpak gnucash-on-flatpak] | ||
− | Flatpak is configured via module files in the [https://github.com/Gnucash/gnucash-on-flatpak/tree/master/modules modules directory]. | + | Flatpak is configured via <tt>json</tt> module files mostly in the [https://github.com/Gnucash/gnucash-on-flatpak/tree/master/modules modules directory]. Some components like the <tt>Runtime</tt> platform are configured in other places. See their full list in [[Release Process#Package the Release on Flathub]]. |
− | Each dependency is defined with a download URL and a SHA checksum. | + | :Each dependency is defined with a download URL and a SHA checksum. |
− | Updating a dependency means updating these URL's with accompanying SHA checskums. | + | :Updating a dependency means updating these URL's with accompanying SHA checskums. |
There are two exceptions: | There are two exceptions: | ||
− | * the gnucash and gnucash-docs version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the [https://flathub.org/apps/details/org.gnucash.GnuCash Flathub GnuCash package] which is built automatically from [https://github.com/flathub/org.gnucash.GnuCash gnucash flathub repository]. | + | * the '''gnucash''' and '''gnucash-docs''' version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the ''download server'' [https://flathub.org/apps/details/org.gnucash.GnuCash Flathub GnuCash package] which is built automatically from the ''working'' [https://github.com/flathub/org.gnucash.GnuCash gnucash flathub repository]. |
− | * the list of cpan modules in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/modules/finance-quote-sources.json finance-quote-sources.json] are generated using a script from [https://github.com/flatpak/flatpak-builder-tools flatpak-builder-tools]. Below is a short description on how this can be generated. | + | * the list of '''cpan modules''' in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/modules/finance-quote-sources.json finance-quote-sources.json] are generated using a script from [https://github.com/flatpak/flatpak-builder-tools flatpak-builder-tools]. Below is a short description on how this can be generated. |
=== How to generate finance-quote-sources.json === | === How to generate finance-quote-sources.json === | ||
− | |||
* Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be<Syntaxhighlight lang="sh"> | * Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be<Syntaxhighlight lang="sh"> | ||
sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'</Syntaxhighlight> | sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'</Syntaxhighlight> | ||
Line 29: | Line 31: | ||
$FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote</Syntaxhighlight> | $FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote</Syntaxhighlight> | ||
− | + | ;Caveat: The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue. | |
+ | === Pull Requests at Flathub === | ||
+ | Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — [{{URL:Gnome|release}}calendar GNOME schedule]. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak. | ||
+ | ;Alternative: <tt>git format-patch</tt> in flatpak.org and <tt>git am</tt> in gnucash-on-flatpak. | ||
==Apple MacOS== | ==Apple MacOS== | ||
Line 45: | Line 50: | ||
Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both. | Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both. | ||
;Update workflow: | ;Update workflow: | ||
− | : | + | :Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) '''aqbanking.de''' displays ''sha256'' checksums, but not the full path. |
− | :# | + | :#;Older versions only: They show only ''MD5'' checksums, so you have to |
− | :# | + | :#:# Download the .tar.gz file, |
− | :# Compare <syntaxhighlight lang="sh" inline>md5sum *.tar.gz</syntaxhighlight> with the website. | + | :#:# Compare <syntaxhighlight lang="sh" inline>md5sum *.tar.gz</syntaxhighlight> with the website. |
− | :# Copy the output from <syntaxhighlight lang="sh" inline>sha256sum *.tar.gz</syntaxhighlight> into '''Flatpak''''s ''sha''. | + | :#:# Copy the output from <syntaxhighlight lang="sh" inline>sha256sum *.tar.gz</syntaxhighlight> into '''Flatpak''''s ''sha''. |
+ | :# Copy the ''URL'' and the ''SHA256 checksum'' into your '''Flatpak''' repository —in Redmine each file version has a unique file number directory— with | ||
+ | :#;URL: <tt>https://aquamaniac.de/rdm/attachments/'''download/'''<serialnumber>/<package></tt> | ||
+ | :# Copy the relevant parts into <branch ''module'' and ''version''> of <tt>gnucash.modules</tt> in your '''Windows''' repository. We maintain no '''OSX''' nightlies! | ||
:# In the '''commit message''' add an abstract of affected modules: <tt>aqhbci</tt>, <tt>ofxdc</tt>, ... from aqbankings git. | :# In the '''commit message''' add an abstract of affected modules: <tt>aqhbci</tt>, <tt>ofxdc</tt>, ... from aqbankings git. | ||
− | :# Create '''PR'''s on ''gnucash-on-flatpak'' | + | :# Create '''PR'''s on ''gnucash-on-flatpak'' and ''gnucash-on-windows'' all with the ''same commit message''. |
+ | :# After the change was committed update the <tt>* version nightly</tt> templates: | ||
+ | {| class="wikitable" | ||
+ | |- | ||
+ | ! scope="col"| Address | ||
+ | ! scope="col"| Content | ||
+ | ! scope="col"| Stable | ||
+ | ! scope="col"| Note | ||
+ | |||
+ | |- style="vertical-align:center;" | ||
+ | |[[Template:AqB Version nightly]] Win | ||
+ | |{{AqB Version nightly}} | ||
+ | |rowspan="2"|{{AqB Version}} | ||
+ | |- style="vertical-align:top;" | ||
+ | |[[Template:AqB Version nightly FP]] | ||
+ | |{{AqB Version nightly FP}} | ||
+ | | | ||
+ | |||
+ | |- style="vertical-align:top;" | ||
+ | |[[Template:FQ Version nightly]] | ||
+ | |{{FQ Version nightly}} | ||
+ | |{{FQ Version}} | ||
+ | | | ||
+ | |} | ||
+ | |||
;Bundling unstable dependencies: Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster. | ;Bundling unstable dependencies: Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster. | ||
+ | |||
+ | ==Continuous Integration== | ||
+ | Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in | ||
+ | ;Program: {{URL:GH}}Gnucash/gnucash/tree/maint/.github/workflows and | ||
+ | :{{URL:GH}}Gnucash/gnucash/tree/maint/util/ci | ||
+ | ;Documentation:{{URL:GH}}Gnucash/gnucash-docs/tree/maint/.github/workflows |
Latest revision as of 11:32, 2 July 2024
GnuCash provides the following binary releases. When feasible and after testing, they can be updated to provide the new version. While the canonical repositories are on code, those are accessible only to the core team so the links are to the Github mirrors.
Keep in mind that these binary distributions are for ordinary users. As a general rule only stable release dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.
- Developers with push privileges
- unless you're able to
- test the new dependency version yourself,
- the upgrade is stable, and
- you're certain that there won't be a negative impact on users
- create a pull request on Github so that others in the core team can test the change.
Dependency Sources has a list of URLs to check for updates.
Contents
Linux Flatpak
Flatpak is configured via json module files mostly in the modules directory. Some components like the Runtime platform are configured in other places. See their full list in Release Process#Package the Release on Flathub.
- Each dependency is defined with a download URL and a SHA checksum.
- Updating a dependency means updating these URL's with accompanying SHA checskums.
There are two exceptions:
- the gnucash and gnucash-docs version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the download server Flathub GnuCash package which is built automatically from the working gnucash flathub repository.
- the list of cpan modules in finance-quote-sources.json are generated using a script from flatpak-builder-tools. Below is a short description on how this can be generated.
How to generate finance-quote-sources.json
- Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be
sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'
- Run the generator script
FBTDIR=<local-path-to-flatpak-builder-tools-repo> GOFDIR=<local-path-to-gnucash-on-flatpak-repo> $FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote
- Caveat
- The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.
Pull Requests at Flathub
Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — GNOME schedule. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak.
- Alternative
- git format-patch in flatpak.org and git am in gnucash-on-flatpak.
Apple MacOS
- Repos, filenames and versions are stored in gnucash-on-osx/blob/master/modulesets/gnucash.modules
Microsoft Windows
- Repos, filenames and versions are stored in gnucash-on-windows/blob/master/gnucash.modules,
- on changes of major versions, adjust the include dirs in bundle-mingw64.ps1.
Nightlies
Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.
- Update workflow
- Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) aqbanking.de displays sha256 checksums, but not the full path.
- Older versions only
- They show only MD5 checksums, so you have to
- Download the .tar.gz file,
- Compare
md5sum *.tar.gz
with the website. - Copy the output from
sha256sum *.tar.gz
into Flatpak's sha.
- Copy the URL and the SHA256 checksum into your Flatpak repository —in Redmine each file version has a unique file number directory— with
- URL
- https://aquamaniac.de/rdm/attachments/download/<serialnumber>/<package>
- Copy the relevant parts into <branch module and version> of gnucash.modules in your Windows repository. We maintain no OSX nightlies!
- In the commit message add an abstract of affected modules: aqhbci, ofxdc, ... from aqbankings git.
- Create PRs on gnucash-on-flatpak and gnucash-on-windows all with the same commit message.
- After the change was committed update the * version nightly templates:
Address | Content | Stable | Note |
---|---|---|---|
Template:AqB Version nightly Win | 6.5.12beta | 6.5.4 | |
Template:AqB Version nightly FP | 6.5.12beta | ||
Template:FQ Version nightly | 1.63 | 1.63 |
- Bundling unstable dependencies
- Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.
Continuous Integration
Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in