Difference between revisions of "Dependency Updates"

From GnuCash
Jump to: navigation, search
(Apple MacOS: Repos, filenames and versions are stored in …)
(Pull Requests at Flathub: GNOME schedule)
 
(18 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
'''Keep in mind that these binary distributions are for ordinary users.''' As a general rule only '''''stable release''''' dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.
 
'''Keep in mind that these binary distributions are for ordinary users.''' As a general rule only '''''stable release''''' dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.
  
For developers with push privileges, unless you're able to
+
;Developers with push privileges: unless you're able to
* test the new dependency version yourself,
+
:* test the new dependency version yourself,
* the upgrade is stable, and
+
:* the upgrade is stable, and
* you're certain that there won't be a negative impact on users
+
:* you're certain that there won't be a negative impact on users
''create a pull request'' on Github so that others in the core team can test the change.
+
:''create a pull request'' on Github so that others in the core team can test the change.
 +
 
 +
[[Dependency Sources]] has a list of URLs to check for updates.
 +
[[Category:Packaging]]
  
 
==Linux Flatpak==
 
==Linux Flatpak==
 
[https://github.com/gnucash/gnucash-on-flatpak gnucash-on-flatpak]
 
[https://github.com/gnucash/gnucash-on-flatpak gnucash-on-flatpak]
  
Flatpak is configured via module files in the [https://github.com/Gnucash/gnucash-on-flatpak/tree/master/modules modules directory].  
+
Flatpak is configured via <tt>json</tt> module files mostly in the [https://github.com/Gnucash/gnucash-on-flatpak/tree/master/modules modules directory]. Some components like the <tt>Runtime</tt> platform are configured in other places. See their full list in [[Release Process#Package the Release on Flathub]].
Each dependency is defined with a download URL and a SHA checksum.
+
:Each dependency is defined with a download URL and a SHA checksum.
Updating a dependency means updating these URL's with accompanying SHA checskums.
+
:Updating a dependency means updating these URL's with accompanying SHA checskums.
  
 
There are two exceptions:
 
There are two exceptions:
* the gnucash and gnucash-docs version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the [https://flathub.org/apps/details/org.gnucash.GnuCash Flathub GnuCash package] which is built automatically from [https://github.com/flathub/org.gnucash.GnuCash gnucash flathub repository].
+
* the '''gnucash''' and '''gnucash-docs''' version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the ''download server'' [https://flathub.org/apps/details/org.gnucash.GnuCash Flathub GnuCash package] which is built automatically from the ''working'' [https://github.com/flathub/org.gnucash.GnuCash gnucash flathub repository].
* the list of cpan modules in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/modules/finance-quote-sources.json finance-quote-sources.json] are generated using a script from [https://github.com/flatpak/flatpak-builder-tools flatpak-builder-tools]. Below is a short description on how this can be generated.
+
* the list of '''cpan modules''' in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/modules/finance-quote-sources.json finance-quote-sources.json] are generated using a script from [https://github.com/flatpak/flatpak-builder-tools flatpak-builder-tools]. Below is a short description on how this can be generated.
  
 
=== How to generate finance-quote-sources.json ===
 
=== How to generate finance-quote-sources.json ===
 
 
* Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be<Syntaxhighlight lang="sh">
 
* Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be<Syntaxhighlight lang="sh">
 
sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'</Syntaxhighlight>
 
sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'</Syntaxhighlight>
Line 29: Line 31:
 
$FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote</Syntaxhighlight>
 
$FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote</Syntaxhighlight>
  
There is one caveat: the generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.
+
;Caveat: The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.
  
 +
=== Pull Requests at Flathub ===
 +
Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — [{{URL:Gnome|release}}calendar GNOME schedule]. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak.
 +
;Alternative: <tt>git format-patch</tt> in flatpak.org and <tt>git am</tt> in gnucash-on-flatpak.
  
 
==Apple MacOS==
 
==Apple MacOS==
 
[https://github.com/gnucash/gnucash-on-osx gnucash-on-osx]
 
[https://github.com/gnucash/gnucash-on-osx gnucash-on-osx]
* Repos, filenames and versions are stored in https://github.com/Gnucash/gnucash-on-osx/blob/master/modulesets/gnucash.modules
+
* Repos, filenames and versions are stored in [https://github.com/Gnucash/gnucash-on-osx/blob/master/modulesets/gnucash.modules gnucash-on-osx/blob/master/modulesets/gnucash.modules]
  
 
==Microsoft Windows==
 
==Microsoft Windows==
Line 45: Line 50:
 
Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.
 
Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.
 
;Update workflow:
 
;Update workflow:
:# Download the .tar.gz file because Flatpak uses sha256, but aqbanking.de currently only displays MD5 checksums,
+
:Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) '''aqbanking.de''' displays ''sha256'' checksums, but not the full path.
:# Copy the ''URL'' into your '''Flatpak''' PR (in Redmine each file version has a unique file number directory),
+
:#;Older versions only: They show only ''MD5'' checksums, so you have to
:# Copy the relevant parts into branch.''module'' and ''version'' of your '''Windows''' PR
+
:#:# Download the .tar.gz file,
:# Compare <syntaxhighlight lang="sh" inline>md5sum *.tar.gz</syntaxhighlight> with the website,
+
:#:# Compare <syntaxhighlight lang="sh" inline>md5sum *.tar.gz</syntaxhighlight> with the website.
:# Copy the output from <syntaxhighlight lang="sh" inline>sha256sum *.tar.gz</syntaxhighlight> into '''Flatpak''''s ''sha'',
+
:#:# Copy the output from <syntaxhighlight lang="sh" inline>sha256sum *.tar.gz</syntaxhighlight> into '''Flatpak''''s ''sha''.
:# In the commit message add an abstract of affected modules: <tt>aqhbci</tt>, <tt>ofxdc</tt>, ... from aqbankings git.
+
:# Copy the ''URL'' and the ''SHA256 checksum'' into your '''Flatpak''' repository —in Redmine each file version has a unique file number directory— with
:# Create PRs on gnucash-on-flatpack and gnucash-on-windows.  
+
:#;URL: <tt>https://aquamaniac.de/rdm/attachments/'''download/'''<serialnumber>/<package></tt>
 +
:# Copy the relevant parts into <branch ''module'' and ''version''> of <tt>gnucash.modules</tt> in your '''Windows''' repository. We maintain no  '''OSX''' nightlies!
 +
:# In the '''commit message''' add an abstract of affected modules: <tt>aqhbci</tt>, <tt>ofxdc</tt>, ... from aqbankings git.
 +
:# Create '''PR'''s on ''gnucash-on-flatpak'' and ''gnucash-on-windows'' all with the ''same commit message''.  
 +
:# After the change was committed update the <tt>* version nightly</tt> templates:
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| Address
 +
! scope="col"| Content
 +
! scope="col"| Stable
 +
! scope="col"| Note
 +
 
 +
|- style="vertical-align:center;"
 +
|[[Template:AqB Version nightly]] Win
 +
|{{AqB Version nightly}}
 +
|rowspan="2"|{{AqB Version}}
 +
|- style="vertical-align:top;"
 +
|[[Template:AqB Version nightly FP]]
 +
|{{AqB Version nightly FP}}
 +
|
 +
 
 +
|- style="vertical-align:top;"
 +
|[[Template:FQ Version nightly]]
 +
|{{FQ Version nightly}}
 +
|{{FQ Version}}
 +
|
 +
|}
 +
 
 
;Bundling unstable dependencies: Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.
 
;Bundling unstable dependencies: Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.
 +
 +
==Continuous Integration==
 +
Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in
 +
;Program: {{URL:GH}}Gnucash/gnucash/tree/maint/.github/workflows and
 +
:{{URL:GH}}Gnucash/gnucash/tree/maint/util/ci
 +
;Documentation:{{URL:GH}}Gnucash/gnucash-docs/tree/maint/.github/workflows

Latest revision as of 11:32, 2 July 2024

GnuCash provides the following binary releases. When feasible and after testing, they can be updated to provide the new version. While the canonical repositories are on code, those are accessible only to the core team so the links are to the Github mirrors.

Keep in mind that these binary distributions are for ordinary users. As a general rule only stable release dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.

Developers with push privileges
unless you're able to
  • test the new dependency version yourself,
  • the upgrade is stable, and
  • you're certain that there won't be a negative impact on users
create a pull request on Github so that others in the core team can test the change.

Dependency Sources has a list of URLs to check for updates.

Linux Flatpak

gnucash-on-flatpak

Flatpak is configured via json module files mostly in the modules directory. Some components like the Runtime platform are configured in other places. See their full list in Release Process#Package the Release on Flathub.

Each dependency is defined with a download URL and a SHA checksum.
Updating a dependency means updating these URL's with accompanying SHA checskums.

There are two exceptions:

  • the gnucash and gnucash-docs version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the download server Flathub GnuCash package which is built automatically from the working gnucash flathub repository.
  • the list of cpan modules in finance-quote-sources.json are generated using a script from flatpak-builder-tools. Below is a short description on how this can be generated.

How to generate finance-quote-sources.json

  • Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be
    sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'
    
  • Run the generator script
    FBTDIR=<local-path-to-flatpak-builder-tools-repo>
    GOFDIR=<local-path-to-gnucash-on-flatpak-repo>
    $FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote
    
Caveat
The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.

Pull Requests at Flathub

Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — GNOME schedule. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak.

Alternative
git format-patch in flatpak.org and git am in gnucash-on-flatpak.

Apple MacOS

gnucash-on-osx

Microsoft Windows

gnucash-on-windows

Nightlies

Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.

Update workflow
Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) aqbanking.de displays sha256 checksums, but not the full path.
  1. Older versions only
    They show only MD5 checksums, so you have to
    1. Download the .tar.gz file,
    2. Compare md5sum *.tar.gz with the website.
    3. Copy the output from sha256sum *.tar.gz into Flatpak's sha.
  2. Copy the URL and the SHA256 checksum into your Flatpak repository —in Redmine each file version has a unique file number directory— with
    URL
    https://aquamaniac.de/rdm/attachments/download/<serialnumber>/<package>
  3. Copy the relevant parts into <branch module and version> of gnucash.modules in your Windows repository. We maintain no OSX nightlies!
  4. In the commit message add an abstract of affected modules: aqhbci, ofxdc, ... from aqbankings git.
  5. Create PRs on gnucash-on-flatpak and gnucash-on-windows all with the same commit message.
  6. After the change was committed update the * version nightly templates:
Address Content Stable Note
Template:AqB Version nightly Win 6.5.12beta 6.5.4
Template:AqB Version nightly FP 6.5.12beta
Template:FQ Version nightly 1.63 1.63
Bundling unstable dependencies
Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.

Continuous Integration

Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in

Program
https://github.com/Gnucash/gnucash/tree/maint/.github/workflows and
https://github.com/Gnucash/gnucash/tree/maint/util/ci
Documentation
https://github.com/Gnucash/gnucash-docs/tree/maint/.github/workflows