img-proof (IPA) provides a command line utility to test images in the Public Cloud

Overview

Continuous testing & Linting Documentation Status Py Versions License

img-proof

overview

img-proof (IPA) provides a command line utility to test images in the Public Cloud (AWS, Azure, GCE, etc.).

With img-proof you can now test custom images in a cloud framework agnostic way with one tool and one API. In the first release, img-proof supports the openSUSE and SLES distributions. It also supports the three largest cloud frameworks (AWS, Azure and GCE). However, it is intended to be distribution agnostic and framework transparent so both are easily extensible.

For each distribution there are specific synchronization points that must be provided. These currently include soft reboot and system update. The synch points not only test functionality but also act as dividers to separate distinct sections of a test suite. For example you can run a test to ensure the proper repos exist before and after a system update. The system update synch point will guarantee the order of tests. Speaking of tests, if you're already familiar with Pytest conventions there's no need to learn a whole new unit testing framework. img-proof is written in Python and leverages the Pytest framework through Testinfra.

Installation

To install the package use the following commands as root:

$ zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/<distribution>
$ zypper refresh
$ zypper in python3-img-proof

Requirements

  • boto3
  • apache-libcloud
  • azure-common
  • azure-mgmt-compute
  • azure-mgmt-network
  • azure-mgmt-resource
  • Click
  • paramiko
  • pytest
  • PyYaml
  • testinfra
  • oci

Docs

Tests

img-proof uses the Testinfra package for writing unit tests. Testinfra leverages Pytest and provides modules such as Package, Process and Service to test the state of images. See the Testinfra Docs for more information on writing infrastructure tests.

img-proof currently passes the Pytest option -x (stop on first failure) through as --early-exit. If there's an interest or need for any other options/args please submit an issue to Github.

CLI Overview

The CLI provides multiple subcommands to initiate image testing:

  • img-proof test

    Test image in the given framework using the supplied test files.

  • img-proof results

    Invokes the default show subcommand img-proof results show 1.

  • img-proof results clear

    Clear the results from the history file.

  • img-proof results delete

    Delete the specified history item from the history log.

  • img-proof results list`

    Display list of results history.

  • img-proof results show

    Display the results or log file for a history item.

  • img-proof list

    Print a list of test files or test cases.

Issues/Enhancements

Please submit issues and requests to Github.

Contributing

Contributions to ipa are welcome and encouraged. See CONTRIBUTING for info on getting started.

License

Copyright (c) 2018 SUSE LLC.

Distributed under the terms of GPL-3.0+ license, see LICENSE for details.

Comments
  • Add exclude parameter

    Add exclude parameter

    Adds a new --exclude parameter to img-proof test, that allows to pass a comma-separated list of test expressions to skip during a test run.

    This allows for using of common patterns e.g. test_sles without running certain tests. Before one needed to state an explicit list of test runs with the risk of not including new test runs as they are included into img-proof.

    Stale 
    opened by grisu48 12
  • Need `--resource-group` parameter with `--running-instance-id`

    Need `--resource-group` parameter with `--running-instance-id`

    Currently we do the assumption, that the resource group name is equal to the name of the VM. This isn't always the case.

    1. We should add a parameter --resource-group for azure.
    2. We could allow using full instance id's like /subscriptions/c011786b-59d7-4817-880c-7cd8a6ca4b19/resourceGroups/openqa-suse-de-1ec3f5a05b7c0712/providers/Microsoft.Compute/virtualMachines/openqa-suse-de-1ec3f5a05b7c0712 and parse resource-group and instance name from it.
    opened by cfconrad 7
  • EC2 SSH key use user data

    EC2 SSH key use user data

    Use user data to add SSH public key to instance on EC2 launch when ssh key name is not provided.

    • SSH public key is generated based on private key file.
    • ssh_key_name arg is optional but takes precedence if provided.
    opened by smarlowucf 7
  • Handle the API change in cloud-regionsrv-client

    Handle the API change in cloud-regionsrv-client

    • The cloud-regionsrv-client API for the is_registered() function has changed from expecting an object to expecting a string. Change the test implementation to handle both cases.
    opened by rjschwei 5
  • traceback generated during mash run

    traceback generated during mash run

    ERROR 2019-05-20 20:14:03,773 TestingService Pass[1]: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 636, in test_im age client File "/usr/lib/python3.6/site-packages/ipa/ipa_utils.py", line 262, in get_hos t_key_fingerprint client.get_transport().get_remote_server_key().get_fingerprint() File "/usr/lib/python3.6/site-packages/paramiko/transport.py", line 761, in ge t_remote_server_key raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/mash/services/testing/ipa_helper.py", l ine 95, in ipa_test timeout=ipa_timeout File "/usr/lib/python3.6/site-packages/ipa/ipa_controller.py", line 131, in te st_image return cloud.test_image() File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 644, in test_im age 'An error occurred retrieving host key: %s' % error ipa.ipa_exceptions.IpaCloudException: An error occurred retrieving host key: No existing session

    mash job id: 4f8c81dd-cfd4-4697-be5f-1777c6c4b220

    opened by rjschwei 4
  • repo test

    repo test

    The suse specific repository test should provide more details. With the switch to the modules and the various flavors of repositories, source, debuginfo etc. it has pretty much become impossible to manually verify registration success. IPA provides the necessary functionality. However, the registration test has a pass/fail status leaving the user/verifyer with the original problem. the test shold produce a diff list in the results if the repo data does not match the expected content. This should hopefully allow the person verifying the test results an opportunity to understand the failure and address it.

    opened by rjschwei 4
  • Apache libcloud

    Apache libcloud

    Consider migrating EC2 and Azure to Apache Libcloud to minimize dependencies and make providers more uniform.

    This would also make adding new providers much easier so long as they are supported by libcloud.

    Todo:

    • [x] GCE
    • [x] EC2
    • [ ] Azure
    enhancement 
    opened by smarlowucf 4
  • Fix wrong exclude type

    Fix wrong exclude type

    Allows the exclude parameter to be any iterable and string. This fixes a regression in which passing any exclude parameter crashed the program.

    Fixes https://github.com/SUSE-Enceladus/img-proof/issues/345

    opened by grisu48 3
  • Add grow root test

    Add grow root test

    Set root disk to non-default size of 50GB except in OCI (TBD). And test confirms that the filesystem properly expands.

    I could not find info in OCI SDK docs about changing root disk parameters when launching instances. Issue opened upstream.

    opened by smarlowucf 3
  • Add Red Hat family distro module and Fedora distro module

    Add Red Hat family distro module and Fedora distro module

    This change adds support for testing Red Hat based distributions, and adds a Fedora module for testing Fedora cloud images.

    This PR is the beginning of investigating using img_proof for automatically testing Fedora Cloud Edition images by the Fedora Cloud SIG.

    (cc: @dustymabe, @major)

    opened by Conan-Kudo 3
  • treat uuid always lowercase

    treat uuid always lowercase

    Since https://github.com/torvalds/linux/commit/712ff25450bd01366301eef81c33e865d901e7b7 the uuid is created with lowercase letters.

    I see failing test with SLE15-SP1: https://openqa.suse.de/tests/2340648#step/EC2_test_sles_ec2_uuid_py__test_sles_ec2_uuid/1

    opened by cfconrad 3
  • configparser>=3.5.0b1 distribution was not found and is required by oci

    configparser>=3.5.0b1 distribution was not found and is required by oci

    Hi,

    when installing img-proof from the http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ repository, I get the following error message:

    [email protected]:~> img-proof --version
    Traceback (most recent call last):
      File "/usr/bin/img-proof", line 6, in <module>
        from pkg_resources import load_entry_point
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3123, in <module>
        @_call_aside
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3107, in _call_aside
        f(*args, **kwargs)
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3136, in _initialize_master_working_set
        working_set = WorkingSet._build_master()
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 578, in _build_master
        ws.require(__requires__)
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 895, in require
        needed = self.resolve(parse_requirements(requirements))
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 781, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'configparser>=3.5.0b1' distribution was not found and is required by oci
    

    I followed the installation procedure from the README.md

    zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ PC_Tools_15_2
    zypper ref
    zypper in python3-img-proof
    
    opened by grisu48 1
  • Drop extension from test names

    Drop extension from test names

    If a user supplies the .py extension in a test description yaml file or in command line it would be helpful for img-proof to drop this by default prior to doing the test search. Or it would be nice to provide more info with the test not found message to make it clear that the extension is not needed.

    enhancement 
    opened by smarlowucf 0
  • SLES test suite should wait for initialization

    SLES test suite should wait for initialization

    Currently when running the SLES test suite on BYOS images we might reboot to soon. Prior to reboot img-proof should wait for the initialization code to finish:

    • In Azure this is waagent
    • In EC2 and OCI it's cloud-init-final
    • In GCE it's google-startup-scripts
    enhancement 
    opened by smarlowucf 0
  • Allow reading of results from an archive

    Allow reading of results from an archive

    If ipa handles listing results from an archive then the user should just give us the archive name, i.e.

    ipa results list --archive $PATH_TO_MY_ARCHIVE ipa results show 5 --archive $PATH_TO_MY_ARCHIVE

    And ipa should just extract the file on the fly to get the results.

    Question is should this be to a temp dir which is deleted after the command or should it be cached somehow for subsequent calls?

    enhancement 
    opened by smarlowucf 0
  • Add Config setup endpoint

    Add Config setup endpoint

    It would be a nice enhancement to have a config setup option in CLI. This would help the user setup the ipa config and/or all config files with info for testing. Since a user may not have config setup for ec2utils, azurectl or Google.

    ipa setup

    Series of prompts requesting config data.

    enhancement 
    opened by smarlowucf 0
Releases(v7.0.0)
Chopper: An Automated Security Headers Analyzer

____ _ _ / ___| |__ ___ _ __ _ __ ___ _ __| | | | | '_ \ / _ \| '_ \| '_ \ / _ \ '__| | | |___| | | | (_) |

Kamran Saifullah (Frog Man) 2 Nov 27, 2022
TermPair lets developers securely share and control terminals in real timeπŸ”’

View and control terminals from your browser with end-to-end encryption πŸ”’

Chad Smith 1.5k Jan 05, 2023
🌈 Beautify your command line interfaces.

Basics Install: pip install iridi Usage: import iridi # Create gradient text # iridi.print(message, colors, options) # Ask for input with gradient

Conrad Crawford 39 Oct 20, 2022
πŸ•° The command line tool for scheduling Python scripts

hickory is a simple command line tool for scheduling Python scripts.

Max Humber 146 Dec 07, 2022
A command line interface to interact with the Hypixel api allowing the user to get stats, leaderboards, etc

HyConsole is a way to get data on players and leaderboards from the Hypixel Minecraft server from the command line. Keep in mind I have no a

1 Feb 14, 2022
A simple automation script that logs into your kra account and files your taxes with one command

EASY_TAX A simple automation script that logs into your kra account and files your taxes with one command Currently works for Chrome users. Will creat

leon koech 13 Sep 23, 2021
This is a repository for collecting global custom management extensions for the Django Framework.

Django Extensions Django Extensions is a collection of custom extensions for the Django Framework. Getting Started The easiest way to figure out what

Django Extensions 6k Jan 03, 2023
An interactive cheatsheet tool for the command-line

navi An interactive cheatsheet tool for the command-line. navi allows you to browse through cheatsheets (that you may write yourself or download from

Denis Isidoro 12.2k Dec 31, 2022
Python CLI script to solve wordles.

Wordle Solver Python CLI script to solve wordles. You need at least python 3.8 installed to run this. No dependencies. Sample Usage Let's say the word

Rachel Brindle 1 Jan 16, 2022
A simple cli utility for importing or exporting dashboard json definitions using the Grafana HTTP API.

P4CMD 🌴 A Python Perforce package that doesn't bring in any other packages to work. Relies on p4cli installed on the system. p4cmd The p4cmd module h

Beam Connectivity 31 Jan 06, 2023
A simple python script to execute a command when a YubiKey is disconnected

YubiKeyExecute A python script to execute a command when a YubiKey / YubiKeys are disconnected. β€β€β€Ž β€Ž How to use: 1. Download the latest release and d

6 Mar 12, 2022
Program Command Line Interface (CLI) Sederhana: Pemesanan Nasi Goreng Hekel

Program ini merupakan aplikasi yang berjalan di dalam command line (terminal). Program ini menggunakan built-in library python yaitu argparse yang dapat menerima parameter saat program ini dijalankan

Habib Abdurrasyid 5 Nov 19, 2021
Color preview command-line tool written in python

Color preview command-line tool written in python

Arnau 1 Dec 27, 2021
A multipurpose discord bot with more than 220 commands

Welcome WM Bot A advanced bot with more than 220 commands to fit your needs Explore the commands Β» View Demo Β· Report Bug Β· Request Feature Table of C

Wasi Master 12 Dec 16, 2022
🎈 `st` is a CLI to quickly kick-off your new Streamlit project

🎈 st - a friendly Streamlit CLI st is a CLI that helps you kick-off a new Streamlit project so you can start crafting the app as soon as possible! Ho

Arnaud 18 Dec 19, 2022
Command line parser for common log format (Nginx default).

Command line parser for common log format (Nginx default).

Lucian Marin 138 Dec 19, 2022
eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.

Command line utilities for tabular data files This is a set of command line utilities for manipulating large tabular data files. Files of numeric and

eBay 1.4k Jan 09, 2023
a GUI app base on warp-cli for linux

warp cloudflare gui a GUI app base on warp-cli for linux Installation read warp-cli install doc. install warp-cli and register with $ warp-cli registe

Moein Aghamirzaei 58 Jan 01, 2023
A command line utility for tracking a stock market portfolio. Primarily featuring high resolution braille graphs.

A command line stock market / portfolio tracker originally insipred by Ericm's Stonks program, featuring unicode for incredibly high detailed graphs even in a terminal.

Conrad Selig 51 Nov 29, 2022
cmdpxl: a totally practical command-line image editor

cmdpxl: a totally practical command-line image editor

Jieruei Chang 476 Jan 07, 2023