Proposal

The user management previously implemented in this charm is unnecessary, and has been removed. This PR is separate from the rest of the k8s charm update work to make it easier to review.

Context

This code was written when I had a worse understanding of the intended purpose of this charm. Postgres/pgbouncer users shouldn't be managed directly by the pgbouncer charm user; instead, they should be derived from the postgres charm relation. Therefore, this code is unnecessary.

The failing integration tests are due to the docker image not being available on github; a future PR will fix this.

Release Notes

• Removed the following user management actions, as well as their associated tests:
  • change_password
  • add_user
  • remove_user
  • get_users

Testing

• Removed irrelevant unit & integration tests. Remaining unit & integration tests still pass.
• Manually tested build & deploy stage

• Added basic pgbouncer charm based on pgbouncer container.
  • Since this container hasn't been pushed anywhere, the integration tests are failing. These should run fine once this is rectified, but in the meantime they pass locally. To run them locally, there should be instructions in contributing.md.
• Added unit tests, coverage is now at 97%
• added basic config file management.
  • These files are necessary for pgbouncer to run, but especially in the case of pgbouncer.ini, they can get very complex. Generating these more complex files will be updated in future PRs.
  • More encryption options for userlist.txt will be added in a future PR.
• added user management actions.
• Added some small amount of docs.

Apologies for the size of this PR; future ones should be smaller and more incremental, and shouldn't have a bunch of user management stuff grafted on.

Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

🚦 To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.

Detected Package Files

• .github/workflows/ci.yaml (github-actions)
• .github/workflows/release.yaml (github-actions)
• requirements.txt (pip_requirements)

Configuration Summary

Based on the default config's presets, Renovate will:

• Start dependency updates only once this onboarding PR is merged
• Enable Renovate Dependency Dashboard creation.
• If Renovate detects semantic commits, it will use semantic commit type fix for dependencies and chore for all others.
• Ignore node_modules, bower_components, vendor and various test/tests directories.
• Autodetect whether to pin dependencies or maintain ranges.
• Rate limit PR creation to a maximum of two per hour.
• Limit to maximum 10 open PRs at any time.
• Group known monorepo packages together.
• Use curated list of recommended non-monorepo package groupings.
• A collection of workarounds for known problems with packages.

🔡 Would you like to change the way Renovate is upgrading your dependencies? Simply edit the renovate.json in this branch with your custom config and the list of Pull Requests in the "What to Expect" section below will be updated the next time Renovate runs.

What to Expect

With your current configuration, Renovate will create 4 Pull Requests:

🚸 Branch creation will be limited to maximum 2 per hour, so it doesn't swamp any CI resources or spam the project. See docs for prhourlylimit for details.

❓ Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.

This PR has been generated by Mend Renovate. View repository job log here.

Proposal

The pgbouncer charm lib previously lived on the VM charm. Since I'm planning to use the k8s charm for development moving forward, porting implementation over to the VM charm once it's fully implemented, tested, and reviewed, it makes sense to keep the charm library on the k8s charm as well.

Context

• I'm planning to use the k8s charm for development because it builds and deploys faster, making testing and development faster.
• All this code has already been reviewed on the VM charm, I'm just changing the names of things and copying unit tests over.

Release Notes

• Rename pgb lib and copy unit tests over from VM charm.

Testing

• Unit and integration tests have run

Proposal

When we restart the k8s controller on which a PGB operator runs, we lose our containers and therefore our config. This PR checks the peer databag for config files if they aren't available in the filesystem.

Context

• We can't keep logs because they aren't stored anywhere, and if they were, they should probably live in COS.

Release Notes

• Check peer databag for config files if they aren't available in filesystem.

Testing

• Added an integration test that restarts the juju controller pod.

Avoid a long-running integration test in case of failing gatekeeping tests. It will slightly increase the complete tests scope runtime but will save (a lot?) of electricity/money for Canonical as often new pull requests have some initial typos/issues to be polished.

Proposal

DPE-781 Run integration tests for passed lint/unit tests only

Context

Costs optimizations for long tests execution (c) John.

Release Notes

Run integration tests for passed lint/unit tests only.

Testing

Tested by GitHub action only.

Proposal

Jira issue: DPE-560 PgBouncer must use encrypted connections when PostgreSQL has TLS enabled.

Context

• This is the same implementation that was made on https://github.com/canonical/pgbouncer-operator/pull/22.

• The difference is that the logs are retrieved using /charm/bin/pebble logs -n=all instead of journalctl -u patroni.service on tests/integration/relations/test_backend_database.py.

• Also, some typing errors were fixed, similarly to https://github.com/canonical/pgbouncer-operator/pull/25.

Release Notes

• Enable TLS encrypted connection between PgBouncer and PostgreSQL.

Testing

• tests/integration/relations/test_backend_database.py contains an additional integration test that checks that TLS is being used.

Proposal

Add new client relations

Context

• This PR is huge, but ~1100 lines of it is libs, lib updates, and the integration test charm. You can skip the following files:
  • everything under lib/
  • tests/integration/relations/pgbouncer_provider/application-charm/lib/charms/data_platform_libs/v0/database_requires.py
• Read pgbouncer_provider.py before reviewing other charm code - the changes added in this file are the reason for the changes in the other files, so reading it first should provide context for the other changes.
• The following PRs implement the same logic in the VM charm, broken into more manageable chunks:
  • https://github.com/canonical/pgbouncer-operator/pull/32
  • https://github.com/canonical/pgbouncer-operator/pull/33
  • https://github.com/canonical/pgbouncer-operator/pull/34

Release Notes

• Add new fully tested client relation
• Update CI
• Update docs
• Fix leader-deletion bug
• Better backend health checks
• General stabilisation work, including updating connection info on pretty much every hook event.
• Brought k8s charm more in line with VM charm design.

Testing

• Updated CI
• Added relevant integration tests & unit tests
  • This includes testing new relation and legacy relation simultaneously

Proposal

Fixes endpoints to point to the correct units when scaling.

Context

I forgot to update how master/standby endpoints are updated when adding scaling. This PR adds that in.

Testing

Updated unit tests, integration tests all pass fine.

Proposal

Jira issue: DPE-472 This PR allows PgBouncer to scale to multiple units.

Context

• We're storing everything in the peer databag for now, including usernames and passwords for the db relations. These will be removed to Juju secrets once they exist.
• The config is stored as an ini file in the databag, rather than a json file, because it's an existing string representation of the data and it's more readable (although there's a bug in jhack that doesn't display the headers)

Release Notes

• added pgb_peers relation and corresponding integration test run
• updated integration test runners to use juju version 2.9.29 to work around scale-down bug.
• added juju get_secret interface

Testing

• updated unit tests
• updated the existing db integration test to work with scaled pgbouncer
  • db-admin test has not been updated because discourse behaves weirdly

Proposal

The install hook isn't well suited for kubernetes charms, so replace it with start.

Release Notes

• replace on-install with on-start
• moved integration test helper functions to integration directory.

Testing

• Integration and unit tests run locally and on CI

Issue

As the ubuntu-latest GH runner image (it's still being updated on other places) was updated to jammy, the step that publishes the charm broke (due to using --destructive-mode).

We can see the issue on https://github.com/canonical/pgbouncer-k8s-operator/actions/runs/3687788578/jobs/6255386375.

Solution

Pin focal runner image (ubuntu-20.04) on release workflow.

Context

The bases on charmcraft.yaml weren't updated to jammy as there are some issues related to some python wheels that should be solved when the work to move the charm to jammy is done.

Testing

Tested sudo charmcraft pack --destructive-mode --quiet manually on a focal VM.

Release Notes

Pin focal runner image on release workflow.