Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Overview

Stream Framework

Build Status StackShare

Activity Streams & Newsfeeds

Examples of what you can build

Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassandra and/or Redis. If you're not using python have a look at Stream, which supports Node, Ruby, PHP, Python, Go, Scala, Java and REST.

Examples of what you can build are:

  • Activity streams such as seen on Github
  • A Twitter style newsfeed
  • A feed like Instagram/ Pinterest
  • Facebook style newsfeeds
  • A notification system

(Feeds are also commonly called: Activity Streams, activity feeds, news streams.)

Stream

Build scalable newsfeeds and activity streams using getstream.io

Stream Framework's authors also offer a web service for building scalable newsfeeds & activity streams at Stream. It allows you to create your feeds by talking to a beautiful and easy to use REST API. There are clients available for Node, Ruby, PHP, Python, Go, Scala and Java. The Get Started page explains the API & concept in a few clicks. It's a lot easier to use, free up to 3 million feed updates and saves you the hassle of maintaining Cassandra, Redis, Faye, RabbitMQ and Celery workers.

Background Articles

A lot has been written about the best approaches to building feed based systems. Here's a collection of some of the talks:

Stream Framework

Installation

Installation through pip is recommended::

$ pip install stream-framework

By default stream-framework does not install the required dependencies for redis and cassandra:

Install stream-framework with Redis dependencies

$ pip install stream-framework[redis]

Install stream-framework with Cassandra dependencies

$ pip install stream-framework[cassandra]

Install stream-framework with both Redis and Cassandra dependencies

$ pip install stream-framework[redis,cassandra]

Authors & Contributors

Resources

Example application

We've included a Pinterest-like example application based on Stream Framework.

Tutorials

Using Stream Framework

This quick example will show you how to publish a "Pin" to all your followers. So let's create an activity for the item you just pinned.

from stream_framework.activity import Activity


def create_activity(pin):
    activity = Activity(
        pin.user_id,
        PinVerb,
        pin.id,
        pin.influencer_id,
        time=make_naive(pin.created_at, pytz.utc),
        extra_context=dict(item_id=pin.item_id)
    )
    return activity

Next up we want to start publishing this activity on several feeds. First of all, we want to insert it into your personal feed, and then into your followers' feeds. Let's start by defining these feeds.

from stream_framework.feeds.redis import RedisFeed


class UserPinFeed(PinFeed):
    key_format = 'feed:user:%(user_id)s'


class PinFeed(RedisFeed):
    key_format = 'feed:normal:%(user_id)s'

Writing to these feeds is very simple. For instance to write to the feed of user 13 one would do:

feed = UserPinFeed(13)
feed.add(activity)

But we don't want to publish to just one users feed. We want to publish to the feeds of all users which follow you. This action is called a "fanout" and is abstracted away in the manager class. We need to subclass the Manager class and tell it how we can figure out which users follow us.

from stream_framework.feed_managers.base import Manager


class PinManager(Manager):
    feed_classes = dict(
        normal=PinFeed,
    )
    user_feed_class = UserPinFeed

    def add_pin(self, pin):
        activity = pin.create_activity()
        # add user activity adds it to the user feed, and starts the fanout
        self.add_user_activity(pin.user_id, activity)

    def get_user_follower_ids(self, user_id):
        ids = Follow.objects.filter(target=user_id).values_list('user_id', flat=True)
        return {FanoutPriority.HIGH:ids}

manager = PinManager()

Now that the manager class is set up, broadcasting a pin becomes as easy as:

manager.add_pin(pin)

Calling this method will insert the pin into your personal feed and into all the feeds of users which follow you. It does so by spawning many small tasks via Celery. In Django (or any other framework) you can now show the users feed.

# django example

@login_required
def feed(request):
    '''
    Items pinned by the people you follow
    '''
    context = RequestContext(request)
    feed = manager.get_feeds(request.user.id)['normal']
    activities = list(feed[:25])
    context['activities'] = activities
    response = render_to_response('core/feed.html', context)
    return response

This example only briefly covered how Stream Framework works. The full explanation can be found on the documentation.

Features

Stream Framework uses Celery and Redis/Cassandra to build a system with heavy writes and extremely light reads. It features:

  • Asynchronous tasks (All the heavy lifting happens in the background, your users don't wait for it)
  • Reusable components (You will need to make tradeoffs based on your use cases, Stream Framework doesn't get in your way)
  • Full Cassandra and Redis support
  • The Cassandra storage uses the new CQL3 and Python-Driver packages, which give you access to the latest Cassandra features.
  • Build for the extremely performant Cassandra 2.1. 2.2 and 3.3 also pass the test suite, but no production experience.
Owner
Thierry Schellenbach
Author Stream-Framework, Founder/CEO getstream.io (Techstars NYC)
Thierry Schellenbach
An extremely fast JavaScript and CSS bundler and minifier

Website | Getting started | Documentation | Plugins | FAQ Why? Our current build tools for the web are 10-100x slower than they could be: The main goa

Evan Wallace 34.2k Jan 04, 2023
☄️ Google Forms autofill script

lazrr 'Destroy Them With Lazers' - Knife Party, 2011 Google Forms autofill script Installation: pip3 install -r requirements.txt Usage: python3 lazrr.

Serezha Rakhmanov 12 Jun 04, 2022
A collection of models, views, middlewares, and forms to help secure a Django project.

Django-Security This package offers a number of models, views, middlewares and forms to facilitate security hardening of Django applications. Full doc

SD Elements 258 Jan 03, 2023
Exemplo de biblioteca com Django

Bookstore Exemplo de biblioteca feito com Django. Este projeto foi feito com: Python 3.9.7 Django 3.2.8 Django Rest Framework 3.12.4 Bootstrap 4.0 Vue

Regis Santos 1 Oct 28, 2021
Django Livre Bank

Django Livre Bank Projeto final da academia Construdelas. API de um banco fictício com clientes, contas e transações. Integrantes da equipe Bárbara Sa

Cecília Costa 3 Dec 22, 2021
RedisTimeSeries python client

redistimeseries-py Deprecation notice As of redis-py 4.0.0 this library is deprecated. It's features have been merged into redis-py. Please either ins

98 Dec 08, 2022
A feature flipper for Django

README Django Waffle is (yet another) feature flipper for Django. You can define the conditions for which a flag should be active, and use it in a num

950 Dec 26, 2022
A fresh approach to autocomplete implementations, specially for Django. Status: v3 stable, 2.x.x stable, 1.x.x deprecated. Please DO regularely ping us with your link at #yourlabs IRC channel

Features Python 2.7, 3.4, Django 2.0+ support (Django 1.11 (LTS), is supported until django-autocomplete-light-3.2.10), Django (multiple) choice suppo

YourLabs 1.7k Jan 01, 2023
This is a basic Todo Application API using Django Rest Framework

Todo Application This is a basic Todo Application API using Django Rest Framework. Todo Section - User can View his previously added todo items, creat

Atharva Parkhe 1 Aug 09, 2022
Run Django tests with testcontainers.

django-rdtwt (Run Django Tests With Testcontainers) This targets users who wish to forget setting up a database for tests. There's no manually startin

2 Jan 09, 2022
Full featured redis cache backend for Django.

Redis cache backend for Django This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidel

Jazzband 2.5k Jan 03, 2023
This is django-import-export module that exports data into many formats

django-import-export This is django-import-export module which exports data into many formats, you can implement this in your admin panel. - Dehydrat

Shivam Rohilla 3 Jun 03, 2021
A Student/ School management application built using Django and Python.

Student Management An awesome student management app built using Django.! Explore the docs » View Demo · Report Bug · Request Feature Table of Content

Nishant Sethi 1 Feb 10, 2022
Mobile Detect is a lightweight Python package for detecting mobile devices (including tablets).

Django Mobile Detector Mobile Detect is a lightweight Python package for detecting mobile devices (including tablets). It uses the User-Agent string c

Botir 6 Aug 31, 2022
Create a netflix-like service using Django, React.js, & More.

Create a netflix-like service using Django. Learn advanced Django techniques to achieve amazing results like never before.

Coding For Entrepreneurs 67 Dec 08, 2022
Add infinite scroll to any django app.

django-infinite-scroll Add infinite scroll to any django app. Features - Allows to add infinite scroll to any page.

Gustavo Teixeira 1 Dec 26, 2021
Helps working with singletons - things like global settings that you want to edit from the admin site.

Django Solo +---------------------------+ | | | | | \ | Django Solo helps

Sylvain Toé 726 Jan 08, 2023
Django + AWS Elastic Transcoder

Django Elastic Transcoder django-elastic-transcoder is an Django app, let you integrate AWS Elastic Transcoder in Django easily. What is provided in t

StreetVoice 66 Dec 14, 2022
django-tables2 - An app for creating HTML tables

django-tables2 - An app for creating HTML tables django-tables2 simplifies the task of turning sets of data into HTML tables. It has native support fo

Jan Pieter Waagmeester 1.6k Jan 03, 2023
A django model and form field for normalised phone numbers using python-phonenumbers

django-phonenumber-field A Django library which interfaces with python-phonenumbers to validate, pretty print and convert phone numbers. python-phonen

Stefan Foulis 1.3k Dec 31, 2022