Python script to extract news from RSS feeds and save it as json.

Related tags

JSONrsscrape
Overview

rsscrape

Python script to extract news from RSS feeds and save it as json.

Usage

$ python3 rsscrape.py
[INFO] Found 51 in 'feeds.txt'
[INFO] Requests 51 XMLs content
[INFO] Scrape 10 items
[INFO] Write 1250 json files to './items'
[INFO] 1648 json files in './items'

Generates a directory items with the results:

./items
    0a1c2b2da6e40ab4e54b8247bbbc1422.json
    fc8ddcf4cc0725bfa35564fb19e4a407.json
    fe15bf1383c382101984ea4fdc6a33ae.json
    ...

Each json file correspondends to a single RSS item:

USA: Corona, war da was? https://de.nachrichten.yahoo.com/usa-corona-war-135203870.html 2021-11-23T13:52:03Z ZEIT ONLINE usa-corona-war-135203870.html " } ">
// f8b40f2bb091e41c53eb35528c433d7f.json 
{
    "title": "USA: Corona, war da was?",
    "link": "https://de.nachrichten.yahoo.com/usa-corona-war-135203870.html",
    "pubDate": "2021-11-23T13:52:03Z",
    "source": "ZEIT ONLINE",
    "guid": "usa-corona-war-135203870.html",
    "raw": "
    
     \"
    http://search.yahoo.com/mrss/\">USA: Corona, war da was?
   https://de.nachrichten.yahoo.com/usa-corona-war-135203870.html
    2021-11-23T13:52:03Z
   \"http://www.zeit.de/index\">ZEIT ONLINE
    \"false
    \">usa-corona-war-135203870.html
   
    \"86
    \" url=
    \"https://s.yimg.com/uu/api/res/1.2/_rdWs7VS_33DY3PJWhkh6Q--~B/aD04MTA7dz0xNDQwO2FwcGlkPXl0YWNoeW9u/https://media.zenfs.com/de/zeit_921/2c35cfd59ae80f62a1ecb89623d2a47f
    \" width=
    \"130
    \"/>
    
     \"publishing company
     \"/>
     "
}
    
   
Owner
Alex Trbznk
pythonic
Alex Trbznk
JSON for Modern C++ Release Scripts

JSON for Modern C++ Release Scripts Preparations Install required tools: make install_requirements. Add required keys to config.json (apparently not c

Niels Lohmann 4 Sep 19, 2022
Generate code from JSON schema files

json-schema-codegen Generate code from JSON schema files. Table of contents Introduction Currently supported languages Requirements Installation Usage

Daniele Esposti 30 Dec 23, 2022
A JSON API for returning Godspeak sentences. Based on the works of Terry A Davis (Rest in Peace, King)

GodspeakAPI A simple API for generating random words ("godspeaks"), inspired by the works of Terrence Andrew Davis (Rest In Peace, King). Installation

Eccentrici 3 Jan 24, 2022
JSON Schema validation library

jsonschema A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible. Supported drafts:

Dmitry Dygalo 309 Jan 01, 2023
Atom, RSS and JSON feed parser for Python 3

Atoma Atom, RSS and JSON feed parser for Python 3. Quickstart Install Atoma with pip: pip install atoma

Nicolas Le Manchet 95 Nov 28, 2022
A tools to find the path of a specific key in deep nested JSON.

如何快速从深层嵌套 JSON 中找到特定的 Key #公众号 在爬虫开发的过程中,我们经常遇到一些 Ajax 加载的接口会返回 JSON 数据。

kingname 56 Dec 13, 2022
Define your JSON schema as Python dataclasses

Define your JSON schema as Python dataclasses

62 Sep 20, 2022
Wikidot-forum-dump - Simple Python script that dumps a Wikidot wiki forum into JSON structures.

wikidot-forum-dump Script is partially based on 2stacks by bluesoul: https://github.com/scuttle/2stacks To dump a Wiki's forum, edit config.py and put

ZZYZX 1 Jun 29, 2022
A query expression for extracting data from JSON.

JSONPATH A selector expression for extracting data from JSON. Quickstarts Installation Install the stable version from PYPI. pip install jsonpath-extr

林玮 (Jade Lin) 33 Oct 22, 2022
JSON Interoperability Vulnerability Labs

JSON Interoperability Vulnerability Labs Description These are the companion labs to my research article "An Exploration of JSON Interoperability Vuln

Bishop Fox 168 Dec 25, 2022
cysimdjson - Very fast Python JSON parsing library

Fast JSON parsing library for Python, 7-12 times faster than standard Python JSON parser.

TeskaLabs 235 Dec 29, 2022
Convert your subscriptions csv file into a valid json for Newpipe!

Newpipe-CSV-Fixer Convert your Google subscriptions CSV file into a valid JSON for Newpipe! Thanks to nikcorg for sharing how to convert the CSV into

Juanjo 44 Dec 29, 2022
Convert your JSON data to a valid Python object to allow accessing keys with the member access operator(.)

JSONObjectMapper Allows you to transform JSON data into an object whose members can be queried using the member access operator. Unlike json.dumps in

Owen Trump 4 Jul 20, 2022
Roamtologseq - A script loads a json export of a Roam graph and cleans it up for import into Logseq

Roam to Logseq The script loads a json export of a Roam graph and cleans it up f

Sebastian Pech 4 Mar 07, 2022
A Python tool that parses JSON documents using JsonPath

A Python tool that parses JSON documents using JsonPath

8 Dec 18, 2022
jq for Python programmers Process JSON and HTML on the command-line with familiar syntax.

jq for Python programmers Process JSON and HTML on the command-line with familiar syntax.

Denis Volk 3 Jan 09, 2022
RedisJSON - a JSON data type for Redis

RedisJSON is a Redis module that implements ECMA-404 The JSON Data Interchange Standard as a native data type. It allows storing, updating and fetching JSON values from Redis keys (documents).

3.4k Dec 29, 2022
Random JSON Key:Pair Json Generator

Random JSON Key:Value Pair Generator This simple script take an engish dictionary of words and and makes random key value pairs. The dictionary has ap

Chris Edwards 1 Oct 14, 2021
Package to Encode/Decode some common file formats to json

ZnJSON Package to Encode/Decode some common file formats to json Available via pip install znjson In comparison to pickle this allows having readable

ZINC 2 Feb 02, 2022
Convert Wii UI formats to JSON5 and vice versa

Convert Wii UI formats to JSON5 and vice versa

Pablo Stebler 11 Aug 28, 2022