C/C++ Dependency Analyzer: a rewrite of John Lakos' dep_utils (adep/cdep/ldep) from

Overview

logo

https://travis-ci.org/rakhimov/cppdep.svg?branch=master 'Build status' Code Health

cppdep performs dependency analysis among components/packages/package groups of a large C/C++ project. This is a rewrite of dep_utils(adep/cdep/ldep), which is provided by John Lakos' book "Large-Scale C++ Software Design", Addison Wesley (1996).

Limitations

  • Indirect extern declarations of global variables or functions instead of including the proper component header with the declarations.
  • Embedded dynamic dependencies, such as dynamic loading and configurable internal services.
  • Preprocessing or macro expansion is not performed. Dependency inclusion via preprocessor meta-programming is not handled.
  • Dependency exclusion with C style multi-line comments or macros is not respected.

Requirements

  1. Python 2.7 or 3.4+
  2. NetworkX
  3. pydot
  4. pydotplus
  5. PyYAML
  6. PyKwalify 1.6.0+

The dependencies can be installed with pip.

$ sudo pip install -r requirements.txt

Installation

From the source:

$ ./setup.py install

The latest stable release from PyPi:

$ pip install cppdep

Usage

Create a configuration file that describes the project for analysis. config_schema.yml is given for guidance.

In the root directory of the project with the configuration file, run the following command to generate dependency analysis reports and graphs.

$ cppdep -c /path/to/config/file

More documentation and example configurations can be found in project wiki.

Acknowledgments

  • John Lakos for inventing the analysis and providing dep_utils.
  • Zhichang Yu for rewriting dep_utils into Python.
Comments
  • Behavior specification for anomalous conflicting component files

    Behavior specification for anomalous conflicting component files

    Anomalous (rare, error) case of having "component.c" and "component.cc" or "component.h" and "component.hpp" at the same time trying to define the same component. In other words, files with the same basenames but different extensions in header and/or source groups.

    opened by rakhimov 1
  • Missing 'pydot' dependency fails on networkx nx_pydot

    Missing 'pydot' dependency fails on networkx nx_pydot

    Traceback (most recent call last):
      File "/.../python3env/bin/cppdep", line 11, in <module>
        sys.exit(main())
      File "/.../python3env/lib/python3.5/site-packages/cppdep/__main__.py", line 59, in main
        analysis.analyze(printer, args)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 797, in analyze
        lambda x: isinstance(x, PackageGroup)))
      File "/.../python3env/lib/python3.5/site-packages/cppdep/cppdep.py", line 779, in _analyze
        digraph.write_dot(graph_name)
      File "/.../python3env/lib/python3.5/site-packages/cppdep/graph.py", line 273, in write_dot
        write_dot(self.digraph, file_basename + '.dot')
      File "<decorator-gen-416>", line 2, in write_dot
      File "/.../python3env/lib/python3.5/site-packages/networkx/utils/decorators.py", line 224, in _open_file
        result = func(*new_args, **kwargs)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 54, in write_dot
        P = to_pydot(G)
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 199, in to_pydot
        pydot = _import_pydot()
      File "/.../python3env/lib/python3.5/site-packages/networkx/drawing/nx_pydot.py", line 348, in _import_pydot
        import pydot
    
    bug 
    opened by rakhimov 0
  • Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    Windows: Unicode Escape Error NetworkX, pydotplus and Python 2.7

    If component/project names contain paths as identifiers, the path separator on Windows \ followed by 'U' or 'u' is interpreted as a Unicode literal by pydotplus plotting on Windows with python 2.7.

    This problem does not show up on python 3.

    For the sake of consistency in the report (stable report) and testing, use Unix path separator in Ids.

    bug 
    opened by rakhimov 0
  • Handle 'ipp' template implementation/source files

    Handle 'ipp' template implementation/source files

    Even though templates are only in headers, ipp files could be considered as implementation pair of the interface header. This occurs, for example, in Boost. #29.

    opened by rakhimov 0
  • Redundant 'ldep' printing of cumulative dependencies

    Redundant 'ldep' printing of cumulative dependencies

    Unlike the original ldep, the cppdep is printing the whole cumulative dependencies; that is, the all link time dependencies are printed. This information is visually provided in the graph, and implicitly provided in the original ldep '-l | -L' flags.

    It seems like the current implementation of the cumulative dependency printing is geared towards debugging rather than analysis report.

    Fix this printing by implementing the original behavior of ldep.

    bug 
    opened by rakhimov 0
  • Pairing header and implementation files in different locations

    Pairing header and implementation files in different locations

    Header and implementation files can be located in different directories, e.g., headers in include and implementation in src. The current implementation assumes the same location.

    bug enhancement 
    opened by rakhimov 0
  • Deduce external packages from the include directives w/o filesystem search

    Deduce external packages from the include directives w/o filesystem search

    The package can be deduced from the include directives following the pattern: "<package/header>", e.g., "<boost/any.hpp>". This approach will avoid the relatively expensive lookup of the header from the system. Whether such external header actually exists on the system or not is irrelevant to the dependency analysis.

    This heuristic would make the configuration simpler and more robust for cross-platform work.

    In addition, the standard library headers can be "hard-coded" into the script since these are the most likely to be used/searched by the source files.

    There's a small chance of false positives if the project under analysis happens not to conform to this convention and invents its own packages with the same names as external ones, e.g., boost, qt, libxml.

    enhancement 
    opened by rakhimov 0
  • Double counting of common components in CCD

    Double counting of common components in CCD

    The current implementation overcounts link time dependencies with common components deep in the dependency graph. Consider: A->B->C, A->D->C. CCD(A) must be 4 (4 object files in total.) The current approach overcounts the common object C, so CCD(A) ends up being 5.

    bug 
    opened by rakhimov 0
  • Extended definition for 'Component'

    Extended definition for 'Component'

    Even though John Lakos defined a component as a pair of h and c files, C++ can have template only components residing only in header files (e.g., STL/Boost/etc.). Moreover, some header-only components may contain only inline functions or macros without any need for an implmentation file (e.g., inline math, Boost PPL). For these reason, unpaired header files can be counted as components by default.

    In addition, the implementation file containing the main function of an application could be considered as a component as well (an entry point).

    bug enhancement 
    opened by rakhimov 0
  • Incorrect dependency processing with file basenames

    Incorrect dependency processing with file basenames

    The current implementation uses the basename of the included files as the key in dependency search. This approach ignores the current location of the dependent files and the language include rules, i.e. <header.h> vs. "header.h". Moreover, it results in name conflicts for files that may belong to completely separate projects or packages.

    bug 
    opened by rakhimov 0
  • Compatibility fixes

    Compatibility fixes

    This PR contains some fixes that I needed to get the cppdep tool running in my recent python 3.8 environment. Also handling of errors in the yaml file was adapted a little bit to get more helpful error messages

    opened by jsinge 0
  • Missing graph module

    Missing graph module

    I've just installed this project, tried to run it but I get this error message.

    Traceback (most recent call last):
      File "cppdep.py", line 36, in <module>
        from .graph import Graph
    ModuleNotFoundError: No module named '__main__.graph'; '__main__' is not a package
    
    bug help wanted question 
    opened by nbourre 2
  • Python 3.6 os.path.commonprefix expects List

    Python 3.6 os.path.commonprefix expects List

      File "/.../cppdep/cppdep.py", line 92, in path_common
        path = os.path.commonprefix(paths)
      File "/home/olzhas/temp/pyenv/lib/python3.6/genericpath.py", line 76, in commonprefix
        if not isinstance(m[0], (list, tuple)):
    TypeError: 'set' object does not support indexing
    
    bug 
    opened by rakhimov 1
  • Include Wrangler Analysis

    Include Wrangler Analysis

    Incorporation of include-wrangler analysis

    • header file cost
    • include cost
    • translation unit cost
    • other features

    In collaboration with include-wrangler author @lukedodd

    enhancement 
    opened by rakhimov 0
  • Options to disable/enable warnings

    Options to disable/enable warnings

    • [ ] Duplicate include
    • [ ] Redundant (transitive from the component header) include
    • [ ] Missing a component header (incomplete component)
    • [ ] Failure to locate a header from an include directive
    enhancement 
    opened by rakhimov 0
Releases(0.2.0)
  • 0.2.0(Feb 3, 2017)

    Added

    • Pairing header and implementation files in different locations (#19)
    • Handle 'ipp' template implementation source files (#31)
    • Behavior specification for anomalous conflicting component files (#27)
    • Implement ignore/exclude paths (#23)
    • Accept glob pattern for source paths (#36)
    • Project wiki pages
    • Regex pattern based include directive classification (#22)
    • Deduce external packages from the include directive w/o filesystem search (#18)
    • Handle header files w/o extensions (Boost/STL/Qt/etc.) (#32)
    • Use POSIX path separator in component names (for cross-platform report stability)
    • Configuration file validation against the schema (with PyKwalify)

    Changed

    • pytest instead of nose
    • YAML configuration files instead of XML (#24)

    Removed

    • Implicit single-path alias Package construction

    Fixed

    • Exception leaks out of main()
    • Unicode Escape Error on graph dot on Windows with Python 2.7 (#35)
    • Python3 UnicodeDecodeError for 'utf-8' in source files (#30)
    • Logging: Type Error: not all arguments converted during string formatting (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 6, 2017)

    Added

    • The original ldep '-l|-L' options to print dependencies (#20)
    • '-o' to print reports into a file
    • Warn about duplicate and redundant includes (#13)
    • Extended definition for 'Component' (#7)
    • PEP-257 conformance (#2)
    • PEP-8 conformance (#1)
    • Python 3 support
    • PyPI package
    • XML configuration example and RNG schema
    • Travis CI (Linux, OS X) and AppVeyor CI (Windows) setups

    Changed

    • Differentiate 'paths' into source, include, and alias.
    • Print warnings to stderr instead of stdout (#12)
    • Report Component levels instead of Graph layers (#9)
    • Refactor the procedural design into the object-oriented design (#4)
    • Change '-f' flag into '-c' flag
    • Replace optparse with argparse
    • XML configuration file format

    Removed

    • Redundant printing a list of cumulative dependencies (#20)
    • Indirect missing-header include warnings
    • Global cross-package and cross-package-group component dependency analysis
    • 'details-of-components/--debug' verbosity
    • dot2any.py helper script
    • Manual profiling code (use pyvmmonitor instead)
    • Manual testing code (automated with nosetest)

    Fixed

    • Level 0 External components missing from the report and graph (#21)
    • Incorrect dependency processing with file basenames (#6)
    • Wrong level calculation for cycles (#8)
    • Double counting of common components in CCD calculations (#11)
    • Missing cycles from the Dot graph (#10)
    • Outdated networkx API usage
    Source code(tar.gz)
    Source code(zip)
Owner
Olzhas Rakhimov
Olzhas Rakhimov
Performant type-checking for python.

Pyre is a performant type checker for Python compliant with PEP 484. Pyre can analyze codebases with millions of lines of code incrementally – providi

Facebook 6.2k Jan 07, 2023
Typing-toolbox for Python 3 _and_ 2.7 w.r.t. PEP 484.

Welcome to the pytypes project pytypes is a typing toolbox w.r.t. PEP 484 (PEP 526 on the road map, later also 544 if it gets accepted). Its main feat

Stefan Richthofer 188 Dec 29, 2022
Static type checker for Python

Static type checker for Python Speed Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fas

Microsoft 9.4k Jan 07, 2023
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Instagram 4.1k Jan 02, 2023
Pymwp is a tool for automatically performing static analysis on programs written in C

pymwp: MWP analysis in Python pymwp is a tool for automatically performing static analysis on programs written in C, inspired by "A Flow Calculus of m

Static Analyses of Program Flows: Types and Certificate for Complexity 2 Dec 02, 2022
A very minimalistic python module that lets you track the time your code snippets take to run.

Clock Keeper A very minimalistic python module that lets you track the time your code snippets take to run. This package is available on PyPI! Run the

Rajdeep Biswas 1 Jan 19, 2022
Metrinome is an all-purpose tool for working with code complexity metrics.

Overview Metrinome is an all-purpose tool for working with code complexity metrics. It can be used as both a REPL and API, and includes: Converters to

26 Dec 26, 2022
TidyPy is a tool that encapsulates a number of other static analysis tools and makes it easy to configure, execute, and review their results.

TidyPy Contents Overview Features Usage Docker Configuration Ignoring Issues Included Tools Included Reporters Included Integrations Extending TidyPy

Jason Simeone 33 Nov 27, 2022
fixup: Automatically add and remove python import statements

fixup: Automatically add and remove python import statements The goal is that running fixup my_file.py will automatically add or remove import stateme

2 May 08, 2022
coala provides a unified command-line interface for linting and fixing all your code, regardless of the programming languages you use.

"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." ― John F. Woods coala provides a

coala development group 3.4k Jan 02, 2023
The uncompromising Python code formatter

The Uncompromising Code Formatter “Any color you like.” Black is the uncompromising Python code formatter. By using it, you agree to cede control over

Python Software Foundation 30.7k Dec 28, 2022
CodeAnalysis - Static Code Analysis: a code comprehensive analysis platform

TCA, Tencent Cloud Code Analysis English | 简体中文 What is TCA Tencent Cloud Code A

Tencent 1.3k Jan 07, 2023
Inspects Python source files and provides information about type and location of classes, methods etc

prospector About Prospector is a tool to analyse Python code and output information about errors, potential problems, convention violations and comple

Python Code Quality Authority 1.7k Dec 31, 2022
A Python utility / library to sort imports.

Read Latest Documentation - Browse GitHub Code Repository isort your imports, so you don't have to. isort is a Python utility / library to sort import

Python Code Quality Authority 5.5k Jan 06, 2023
pycallgraph is a Python module that creates call graphs for Python programs.

Project Abandoned Many apologies. I've stopped maintaining this project due to personal time constraints. Blog post with more information. I'm happy t

gak 1.7k Jan 01, 2023
Data parsing and validation using Python type hints

pydantic Data validation and settings management using Python type hinting. Fast and extensible, pydantic plays nicely with your linters/IDE/brain. De

Samuel Colvin 12.1k Jan 05, 2023
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Alex Grönholm 1.1k Dec 19, 2022
An interpreter for the X1 bytecode.

X1 Bytecode Interpreter The X1 Bytecode is bytecode designed for simplicity in programming design and compilation. Bytecode Instructions push

Thanasis Tzimas 1 Jan 15, 2022
Turn your Python and Javascript code into DOT flowcharts

Notes from 2017 This is an older project which I am no longer working on. It was built before ES6 existed and before Python 3 had much usage. While it

Scott Rogowski 3k Jan 09, 2023
A static analysis tool for Python

pyanalyze Pyanalyze is a tool for programmatically detecting common mistakes in Python code, such as references to undefined variables and some catego

Quora 212 Jan 07, 2023