A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Overview


MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-the-art vision and language models and has powered multiple research projects at Facebook AI Research. See full list of project inside or built on MMF here.

MMF is powered by PyTorch, allows distributed training and is un-opinionated, scalable and fast. Use MMF to bootstrap for your next vision and language multimodal research project by following the installation instructions. Take a look at list of MMF features here.

MMF also acts as starter codebase for challenges around vision and language datasets (The Hateful Memes, TextVQA, TextCaps and VQA challenges). MMF was formerly known as Pythia. The next video shows an overview of how datasets and models work inside MMF. Checkout MMF's video overview.

Installation

Follow installation instructions in the documentation.

Documentation

Learn more about MMF here.

Citation

If you use MMF in your work or use any models published in MMF, please cite:

@misc{singh2020mmf,
  author =       {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and
                 Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi},
  title =        {MMF: A multimodal framework for vision and language research},
  howpublished = {\url{https://github.com/facebookresearch/mmf}},
  year =         {2020}
}

License

MMF is licensed under BSD license available in LICENSE file

Comments
  • [feat] Add ViLT model

    [feat] Add ViLT model

    Stack from ghstack:

    • #1142
    • #1106
    • #1103
    • -> #1102
    • #1141
    • #1101

    Add ViLT model and unit tests. ViLT model from megaVLT, using ViT from huggingface.

    Differential Revision: D31117427

    CLA Signed Merged 
    opened by Ryan-Qiyu-Jiang 49
  • [feat] Add ViLT image and text embeddings

    [feat] Add ViLT image and text embeddings

    Stack from ghstack:

    • #1142
    • #1106
    • #1103
    • #1102
    • #1141
    • -> #1101

    Add ViT image and Bert text embedding encoders for ViLT model.

    Differential Revision: D31117428

    CLA Signed Merged 
    opened by Ryan-Qiyu-Jiang 46
  • [feat][PL][8/N] save config with checkpoint

    [feat][PL][8/N] save config with checkpoint

    Stack from ghstack:

    • #1035 [feat][PL][11/N] Inference with test
    • #1033 [feat][PL][10/N] lightning distributed
    • #1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
    • #1024 [feat][PL][8/N] save config with checkpoint
    • #1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update
    • #1022 [feat][PL][6/N] pretrained state mapping for PL

    Differential Revision: D29670421

    CLA Signed Merged 
    opened by ytsheng 39
  • [feat][PL][6/N] pretrained state mapping for PL

    [feat][PL][6/N] pretrained state mapping for PL

    Stack from ghstack:

    • #1035 [feat][PL][11/N] Inference with test
    • #1033 [feat][PL][10/N] lightning distributed
    • #1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
    • #1024 [feat][PL][8/N] save config with checkpoint
    • #1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update
    • #1022 [feat][PL][6/N] pretrained state mapping for PL

    Differential Revision: D29670417

    CLA Signed 
    opened by ytsheng 37
  • [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

    [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

    Stack from ghstack:

    • #1035 [feat][PL][11/N] Inference with test
    • #1033 [feat][PL][10/N] lightning distributed
    • #1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
    • #1024 [feat][PL][8/N] save config with checkpoint
    • #1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update
    • #1022 [feat][PL][6/N] pretrained state mapping for PL

    Differential Revision: D29888294

    CLA Signed 
    opened by ytsheng 36
  • [feat] Add ViLT encoder

    [feat] Add ViLT encoder

    Stack from ghstack:

    • #1142
    • #1106
    • #1103
    • #1102
    • #1141
    • #1101
    • -> #1100

    Add encoder for loading pretrained ViTModel weights from huggingface

    Differential Revision: D31117425

    CLA Signed 
    opened by Ryan-Qiyu-Jiang 33
  • [feat][PL][10/N] lightning distributed

    [feat][PL][10/N] lightning distributed

    Stack from ghstack:

    • #1035 [feat][PL][11/N] Inference with test
    • #1033 [feat][PL][10/N] lightning distributed
    • #1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
    • #1024 [feat][PL][8/N] save config with checkpoint
    • #1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update
    • #1022 [feat][PL][6/N] pretrained state mapping for PL

    Differential Revision: D29981634

    CLA Signed 
    opened by ytsheng 33
  • [feat][PL][7/N] trainer checkpoint from mmf to lightning update

    [feat][PL][7/N] trainer checkpoint from mmf to lightning update

    Stack from ghstack:

    • #1035 [feat][PL][11/N] Inference with test
    • #1033 [feat][PL][10/N] lightning distributed
    • #1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
    • #1024 [feat][PL][8/N] save config with checkpoint
    • #1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update
    • #1022 [feat][PL][6/N] pretrained state mapping for PL

    Differential Revision: D29670422

    CLA Signed 
    opened by ytsheng 33
  • [feat] Add ViT modules

    [feat] Add ViT modules

    Stack from ghstack:

    • #1142
    • #1106
    • #1103
    • #1102
    • #1141
    • #1101
    • #1100
    • -> #1099

    Add modified huggingface ViT modules from megaVLT supporting attention masks

    Differential Revision: D31117429

    CLA Signed 
    opened by Ryan-Qiyu-Jiang 30
  • [feat] Add UNITER model wrapper

    [feat] Add UNITER model wrapper

    Stack from ghstack:

    • #1144
    • #1128
    • -> #1127
    • #1133

    Add UNITER model to mmf registry with support for pretraining through yaml head configs.

    Differential Revision: D31768457

    CLA Signed 
    opened by Ryan-Qiyu-Jiang 28
  • Error during training Textvqa

    Error during training Textvqa

    ❓ Questions and Help

    Hi,

    While I am trying the training code with m4c model, I am getting the following error,

    2021-03-11T03:34:15 | mmf.utils.general: Total Parameters: 90850184. Trained Parameters: 90850184 2021-03-11T03:34:15 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 363, in next data = self._next_data() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data return self._process_data(data) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data data.reraise() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch_utils.py", line 395, in reraise raise self.exc_type(msg) KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 231, in load image_id = int(split.split("")[-1]) ValueError: invalid literal for int() with base 10: 'train\7f14a505b6edcbc5'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataset.py", line 207, in getitem return self.datasets[dataset_idx][sample_idx] File "d:\project\new folder\mmf\mmf\datasets\builders\textvqa\dataset.py", line 100, in getitem features = self.features_db[idx] File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 91, in getitem return self.get(image_info) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 99, in get return self.from_path(feature_path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 107, in from_path features, infos = self._get_image_features_and_info(path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 80, in _get_image_features_and_info image_feats, infos = self._read_features_and_info(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 65, in _read_features_and_info feature, info = feature_reader.read(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 95, in read return self.feat_reader.read(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 158, in read image_info = self._load(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 238, in _load img_id_idx = self.image_id_indices[image_id] KeyError: b'train\7f14a505b6edcbc5'

    When I tried with model = "Lorra", I am getting the below error,

    2021-03-11T03:27:37 | mmf.utils.general: Total Parameters: 192497485. Trained Parameters: 192497485 2021-03-11T03:27:37 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "d:\project\new folder\mmf\mmf\datasets\multi_dataset_loader.py", line 213, in iter return iter(self.loaders[0]) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 291, in iter return _MultiProcessingDataLoaderIter(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 737, in init w.start() File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) BrokenPipeError: [Errno 32] Broken pipe

    Kindly help me to resolve this issue.

    needs more info triaged 
    opened by Mano2610 28
  • Bump terser from 5.10.0 to 5.16.1 in /website

    Bump terser from 5.10.0 to 5.16.1 in /website

    Bumps terser from 5.10.0 to 5.16.1.

    Changelog

    Sourced from terser's changelog.

    v5.16.1

    • Properly handle references in destructurings (const { [reference]: val } = ...)
    • Allow parsing of .#privatefield in nested classes
    • Do not evaluate operations that return large strings if that would make the output code larger
    • Make collapse_vars handle block scope correctly
    • Internal improvements: Typos (#1311), more tests, small-scale refactoring

    v5.16.0

    • Disallow private fields in object bodies (#1011)
    • Parse #privatefield in object (#1279)
    • Compress #privatefield in object

    v5.15.1

    • Fixed missing parentheses around optional chains
    • Avoid bare let or const as the bodies of if statements (#1253)
    • Small internal fixes (#1271)
    • Avoid inlining a class twice and creating two equivalent but !== classes.

    v5.15.0

    • Basic support for ES2022 class static initializer blocks.
    • Add AudioWorkletNode constructor options to domprops list (#1230)
    • Make identity function inliner not inline id(...expandedArgs)

    v5.14.2

    • Security fix for RegExps that should not be evaluated (regexp DDOS)
    • Source maps improvements (#1211)
    • Performance improvements in long property access evaluation (#1213)

    v5.14.1

    • keep_numbers option added to TypeScript defs (#1208)
    • Fixed parsing of nested template strings (#1204)

    v5.14.0

    • Switched to @​jridgewell/source-map for sourcemap generation (#1190, #1181)
    • Fixed source maps with non-terminated segments (#1106)
    • Enabled typescript types to be imported from the package (#1194)
    • Extra DOM props have been added (#1191)
    • Delete the AST while generating code, as a means to save RAM

    v5.13.1

    • Removed self-assignments (varname=varname) (closes #1081)
    • Separated inlining code (for inlining things into references, or removing IIFEs)
    • Allow multiple identifiers with the same name in var destructuring (eg var { a, a } = x) (#1176)

    v5.13.0

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    CLA Signed dependencies javascript 
    opened by dependabot[bot] 0
  • Bump minimist from 1.2.5 to 1.2.7 in /website

    Bump minimist from 1.2.5 to 1.2.7 in /website

    Bumps minimist from 1.2.5 to 1.2.7.

    Changelog

    Sourced from minimist's changelog.

    v1.2.7 - 2022-10-10

    Commits

    • [meta] add auto-changelog 0ebf4eb
    • [actions] add reusable workflows e115b63
    • [eslint] add eslint; rules to enable later are warnings f58745b
    • [Dev Deps] switch from covert to nyc ab03356
    • [readme] rename and add badges 236f4a0
    • [meta] create FUNDING.yml; add funding in package.json 783a49b
    • [meta] use npmignore to autogenerate an npmignore file f81ece6
    • Only apps should have lockfiles 56cad44
    • [Dev Deps] update covert, tape; remove unnecessary tap 49c5f9f
    • [Tests] add aud in posttest 228ae93
    • [meta] add safe-publish-latest 01fc23f
    • [meta] update repo URLs 6b164c7

    v1.2.6 - 2022-03-21

    Commits

    • test from prototype pollution PR bc8ecee
    • isConstructorOrProto adapted from PR c2b9819
    • security notice for additional prototype pollution issue ef88b93
    Commits
    • c590d75 v1.2.7
    • 0ebf4eb [meta] add auto-changelog
    • e115b63 [actions] add reusable workflows
    • 01fc23f [meta] add safe-publish-latest
    • f58745b [eslint] add eslint; rules to enable later are warnings
    • 228ae93 [Tests] add aud in posttest
    • 236f4a0 [readme] rename and add badges
    • ab03356 [Dev Deps] switch from covert to nyc
    • 49c5f9f [Dev Deps] update covert, tape; remove unnecessary tap
    • 783a49b [meta] create FUNDING.yml; add funding in package.json
    • Additional commits viewable in compare view
    Maintainer changes

    This version was pushed to npm by ljharb, a new releaser for minimist since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    CLA Signed dependencies javascript 
    opened by dependabot[bot] 0
  • Bump prismjs from 1.26.0 to 1.29.0 in /website

    Bump prismjs from 1.26.0 to 1.29.0 in /website

    Bumps prismjs from 1.26.0 to 1.29.0.

    Release notes

    Sourced from prismjs's releases.

    v1.29.0

    Release 1.29.0

    v1.28.0

    Release 1.28.0

    v1.27.0

    Release 1.27.0

    Changelog

    Sourced from prismjs's changelog.

    1.29.0 (2022-08-23)

    New components

    Updated components

    Updated plugins

    • Line Highlight
    • Normalize Whitespace

    Other

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    CLA Signed dependencies javascript 
    opened by dependabot[bot] 0
  • Bump eventsource from 1.1.0 to 1.1.2 in /website

    Bump eventsource from 1.1.0 to 1.1.2 in /website

    Bumps eventsource from 1.1.0 to 1.1.2.

    Changelog

    Sourced from eventsource's changelog.

    1.1.2

    • Inline origin resolution, drops original dependency (#281 Espen Hovlandsdal)

    1.1.1

    • Do not include authorization and cookie headers on redirect to different origin (#273 Espen Hovlandsdal)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    CLA Signed dependencies javascript 
    opened by dependabot[bot] 0
  • Bump follow-redirects from 1.14.7 to 1.15.2 in /website

    Bump follow-redirects from 1.14.7 to 1.15.2 in /website

    Bumps follow-redirects from 1.14.7 to 1.15.2.

    Commits
    • 9655237 Release version 1.15.2 of the npm package.
    • 6e2b86d Default to localhost if no host given.
    • 449e895 Throw invalid URL error on relative URLs.
    • e30137c Use type functions.
    • 76ea31f ternary operator syntax fix
    • 84c00b0 HTTP header lines are separated by CRLF.
    • d28bcbf Create SECURITY.md (#202)
    • 62a551c Release version 1.15.1 of the npm package.
    • 7fe0779 Use for ... of.
    • 948c30c Fix redirecting to relative URL when using proxy
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    CLA Signed dependencies javascript 
    opened by dependabot[bot] 0
  • What are the inputs to be provided in pretrained VilBERT model?

    What are the inputs to be provided in pretrained VilBERT model?

    Hello, I loaded pretrained model by the following provided code. from mmf.common.registry import registry

    model_cls = registry.get_model_class("vilbert") model = model_cls.from_pretrained("vilbert.finetuned.hateful_memes.direct")

    Now, I want to use this model on my dataset but I can't figure out what are the inputs to be provided in the model. These all are the inputs to be given, input_0, input_1, targets, input_mask, segment_ids, return_output_dict, image_info_0 , image_features_0

    opened by ApoorvVerma5 0
Releases(v0.3.1)
  • v0.3(Jun 26, 2019)

    Features

    • Multi-tasking support: Multitasking over various datasets available in Pythia
    • Distributed training support
    • Better Customization Support: Use your custom losses, metrics, optimizers and lot of more
    • Standardized Trainer API: A standard trainer API to fit most of your use-cases, if not inherit and build your own trainer
    • Processors: Use processors to build out your datasets easily and without pain
    • SampleList and Sample: Use SampleList and Sample to have more granular control over what you pass and a single unified API for accessing attributes whether inside a dataset, a single sample or a batch
    • Feature Extraction: A new simple script to extract out features and related information from VQA MaskRCNN Benchmark
    • Registry: No need to manually load datasets and models anymore, registry takes care of loading your models, datasets and other classes at the fly. Think of registry as a singleton containing all that you need.
    • Tensorboard Logging: Tensorboard logging is now provided by default.
    • Configuration: Better hierarchal configuration system for better separation of concerns.
    • Checkpointing: Better control checkpointing and resuming
    • Logging: Better logging is provided now with eta, individual val losses and metrics. Just pass them back from your model and everything logs automatically
    • EvalAI Evaluation: Now, directly output JSON files that can be uploaded to EvalAI
    • Early Stopping
    • New Embeddings Support: FastText, GloVe, BERT etc.

    Datasets

    Following new datasets were added:

    Models

    Note: There are a lot of breaking changes in the API from v0.1. Refer to the documentation to learn more on how to work with Pythia v0.3.

    Source code(tar.gz)
    Source code(zip)
Owner
Facebook Research
Facebook Research
RIDE automatically creates the package and boilerplate OOP Python node scripts as per your needs

RIDE: ROS IDE RIDE automatically creates the package and boilerplate OOP Python code for nodes as per your needs (RIDE is not an IDE, but even ROS isn

Jash Mota 20 Jul 14, 2022
Beyond the Imitation Game collaborative benchmark for enormous language models

BIG-bench 🪑 The Beyond the Imitation Game Benchmark (BIG-bench) will be a collaborative benchmark intended to probe large language models, and extrap

Google 1.3k Jan 01, 2023
Coreference resolution for English, German and Polish, optimised for limited training data and easily extensible for further languages

Coreferee Author: Richard Paul Hudson, msg systems ag 1. Introduction 1.1 The basic idea 1.2 Getting started 1.2.1 English 1.2.2 German 1.2.3 Polish 1

msg systems ag 169 Dec 21, 2022
Extract rooms type, door, neibour rooms, rooms corners nad bounding boxes, and generate graph from rplan dataset

Housegan-data-reader House-GAN++ (data-reader) Code and instructions for converting rplan dataset (raster images) to housegan++ data format. House-GAN

Sepid Hosseini 13 Nov 24, 2022
Pretty-doc - Composable text objects with python

pretty-doc from __future__ import annotations from dataclasses import dataclass

Taine Zhao 2 Jan 17, 2022
Yomichad - a Japanese pop-up dictionary that can display readings and English definitions of Japanese words

Yomichad is a Japanese pop-up dictionary that can display readings and English definitions of Japanese words, kanji, and optionally named entities. It is similar to yomichan, 10ten, and rikaikun in s

Jonas Belouadi 7 Nov 07, 2022
Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Tensor2Tensor Tensor2Tensor, or T2T for short, is a library of deep learning models and datasets designed to make deep learning more accessible and ac

12.9k Jan 07, 2023
a CTF web challenge about making screenshots

screenshotter (web) A CTF web challenge about making screenshots. It is inspired by a bug found in real life. The challenge was created by @LiveOverfl

219 Jan 02, 2023
Word2Wave: a framework for generating short audio samples from a text prompt using WaveGAN and COALA.

Word2Wave is a simple method for text-controlled GAN audio generation. You can either follow the setup instructions below and use the source code and CLI provided in this repo or you can have a play

Ilaria Manco 91 Dec 23, 2022
Utilizing RBERT model for KLUE Relation Extraction task

RBERT for Relation Extraction task for KLUE Project Description Relation Extraction task is one of the task of Korean Language Understanding Evaluatio

snoop2head 14 Nov 15, 2022
A programming language with logic of Python, and syntax of all languages.

Pytov The idea was to take all well known syntaxes, and combine them into one programming language with many posabilities. Installation Install using

Yuval Rosen 14 Dec 07, 2022
FireFlyer Record file format, writer and reader for DL training samples.

FFRecord The FFRecord format is a simple format for storing a sequence of binary records developed by HFAiLab, which supports random access and Linux

77 Jan 04, 2023
Code for paper: An Effective, Robust and Fairness-awareHate Speech Detection Framework

BiQQLSTM_HS Code and data for paper: Title: An Effective, Robust and Fairness-awareHate Speech Detection Framework. Authors: Guanyi Mou and Kyumin Lee

Guanyi Mou 2 Dec 27, 2022
Write Python in Urdu - اردو میں کوڈ لکھیں

UrduPython Write simple Python in Urdu. How to Use Write Urdu code in سامپل۔پے The mappings are as following: "۔": ".", "،":

Saad A. Bazaz 26 Nov 27, 2022
Script and models for clustering LAION-400m CLIP embeddings.

clustering-laion400m Script and models for clustering LAION-400m CLIP embeddings. Models were fit on the first million or so image embeddings. A subje

Peter Baylies 22 Oct 04, 2022
GPT-Code-Clippy (GPT-CC) is an open source version of GitHub Copilot, a language model

GPT-Code-Clippy (GPT-CC) is an open source version of GitHub Copilot, a language model -- based on GPT-3, called GPT-Codex -- that is fine-tuned on publicly available code from GitHub.

Nathan Cooper 2.3k Jan 01, 2023
pkuseg多领域中文分词工具; The pkuseg toolkit for multi-domain Chinese word segmentation

pkuseg:一个多领域中文分词工具包 (English Version) pkuseg 是基于论文[Luo et. al, 2019]的工具包。其简单易用,支持细分领域分词,有效提升了分词准确度。 目录 主要亮点 编译和安装 各类分词工具包的性能对比 使用方式 论文引用 作者 常见问题及解答 主要

LancoPKU 6k Dec 29, 2022
PyTorch implementation of the paper: Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding

Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding This repository contains the official PyTorch implementation of th

Xiao Xu 26 Dec 14, 2022
This repository contains helper functions which can help you generate additional data points depending on your NLP task.

NLP Albumentations For Data Augmentation This repository contains helper functions which can help you generate additional data points depending on you

Aflah 6 May 22, 2022
Retraining OpenAI's GPT-2 on Discord Chats

Train OpenAI's GPT-2 on Discord Chats Retraining a Text Generation Model on Discord Chats using gpt-2-simple that wraps existing model fine-tuning and

Ayush Mishra 4 Oct 27, 2022