Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Overview

Stream Framework

Build Status StackShare

Activity Streams & Newsfeeds

Examples of what you can build

Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassandra and/or Redis. If you're not using python have a look at Stream, which supports Node, Ruby, PHP, Python, Go, Scala, Java and REST.

Examples of what you can build are:

  • Activity streams such as seen on Github
  • A Twitter style newsfeed
  • A feed like Instagram/ Pinterest
  • Facebook style newsfeeds
  • A notification system

(Feeds are also commonly called: Activity Streams, activity feeds, news streams.)

Stream

Build scalable newsfeeds and activity streams using getstream.io

Stream Framework's authors also offer a web service for building scalable newsfeeds & activity streams at Stream. It allows you to create your feeds by talking to a beautiful and easy to use REST API. There are clients available for Node, Ruby, PHP, Python, Go, Scala and Java. The Get Started page explains the API & concept in a few clicks. It's a lot easier to use, free up to 3 million feed updates and saves you the hassle of maintaining Cassandra, Redis, Faye, RabbitMQ and Celery workers.

Background Articles

A lot has been written about the best approaches to building feed based systems. Here's a collection of some of the talks:

Stream Framework

Installation

Installation through pip is recommended::

$ pip install stream-framework

By default stream-framework does not install the required dependencies for redis and cassandra:

Install stream-framework with Redis dependencies

$ pip install stream-framework[redis]

Install stream-framework with Cassandra dependencies

$ pip install stream-framework[cassandra]

Install stream-framework with both Redis and Cassandra dependencies

$ pip install stream-framework[redis,cassandra]

Authors & Contributors

Resources

Example application

We've included a Pinterest-like example application based on Stream Framework.

Tutorials

Using Stream Framework

This quick example will show you how to publish a "Pin" to all your followers. So let's create an activity for the item you just pinned.

from stream_framework.activity import Activity


def create_activity(pin):
    activity = Activity(
        pin.user_id,
        PinVerb,
        pin.id,
        pin.influencer_id,
        time=make_naive(pin.created_at, pytz.utc),
        extra_context=dict(item_id=pin.item_id)
    )
    return activity

Next up we want to start publishing this activity on several feeds. First of all, we want to insert it into your personal feed, and then into your followers' feeds. Let's start by defining these feeds.

from stream_framework.feeds.redis import RedisFeed


class UserPinFeed(PinFeed):
    key_format = 'feed:user:%(user_id)s'


class PinFeed(RedisFeed):
    key_format = 'feed:normal:%(user_id)s'

Writing to these feeds is very simple. For instance to write to the feed of user 13 one would do:

feed = UserPinFeed(13)
feed.add(activity)

But we don't want to publish to just one users feed. We want to publish to the feeds of all users which follow you. This action is called a "fanout" and is abstracted away in the manager class. We need to subclass the Manager class and tell it how we can figure out which users follow us.

from stream_framework.feed_managers.base import Manager


class PinManager(Manager):
    feed_classes = dict(
        normal=PinFeed,
    )
    user_feed_class = UserPinFeed

    def add_pin(self, pin):
        activity = pin.create_activity()
        # add user activity adds it to the user feed, and starts the fanout
        self.add_user_activity(pin.user_id, activity)

    def get_user_follower_ids(self, user_id):
        ids = Follow.objects.filter(target=user_id).values_list('user_id', flat=True)
        return {FanoutPriority.HIGH:ids}

manager = PinManager()

Now that the manager class is set up, broadcasting a pin becomes as easy as:

manager.add_pin(pin)

Calling this method will insert the pin into your personal feed and into all the feeds of users which follow you. It does so by spawning many small tasks via Celery. In Django (or any other framework) you can now show the users feed.

# django example

@login_required
def feed(request):
    '''
    Items pinned by the people you follow
    '''
    context = RequestContext(request)
    feed = manager.get_feeds(request.user.id)['normal']
    activities = list(feed[:25])
    context['activities'] = activities
    response = render_to_response('core/feed.html', context)
    return response

This example only briefly covered how Stream Framework works. The full explanation can be found on the documentation.

Features

Stream Framework uses Celery and Redis/Cassandra to build a system with heavy writes and extremely light reads. It features:

  • Asynchronous tasks (All the heavy lifting happens in the background, your users don't wait for it)
  • Reusable components (You will need to make tradeoffs based on your use cases, Stream Framework doesn't get in your way)
  • Full Cassandra and Redis support
  • The Cassandra storage uses the new CQL3 and Python-Driver packages, which give you access to the latest Cassandra features.
  • Build for the extremely performant Cassandra 2.1. 2.2 and 3.3 also pass the test suite, but no production experience.
Owner
Thierry Schellenbach
Author Stream-Framework, Founder/CEO getstream.io (Techstars NYC)
Thierry Schellenbach
Django Girls Tutorial Workshop

Django Girls Tutorial Workshop A log of activities during the workshop. this is an H2 git remote add origin https://github.com/ahuimanu/django_girls_t

Jeffry Babb 1 Oct 27, 2021
This is a Django app that uses numerous Google APIs such as reCAPTURE, maps and waypoints

Django project that uses Googles APIs to auto populate fields, display maps and routes for multiple waypoints

Bobby Stearman 57 Dec 03, 2022
A app for managing lessons with Django

Course Notes A app for managing lessons with Django Some Ideas

Motahhar.Mokfi 6 Jan 28, 2022
Use minify-html, the extremely fast HTML + JS + CSS minifier, with Django.

django-minify-html Use minify-html, the extremely fast HTML + JS + CSS minifier, with Django. Requirements Python 3.8 to 3.10 supported. Django 2.2 to

Adam Johnson 60 Dec 28, 2022
A clone of https://virgool.io written in django

Virgool clone A clone of virgool blog written in django Installation first rename the .env.sample to .env and fill it. with docker docker-compose up -

Danial Selmipoor 7 Dec 23, 2022
Store model history and view/revert changes from admin site.

django-simple-history django-simple-history stores Django model state on every create/update/delete. This app supports the following combinations of D

Jazzband 1.8k Jan 06, 2023
Dynamic Django settings.

Constance - Dynamic Django settings A Django app for storing dynamic settings in pluggable backends (Redis and Django model backend built in) with an

Jazzband 1.5k Jan 07, 2023
Django-gmailapi-json-backend - Email backend for Django which sends email via the Gmail API through a JSON credential

django-gmailapi-json-backend Email backend for Django which sends email via the

Innove 1 Sep 09, 2022
React.JS - Django Application Template

OTS React.JS - DJango Web Application (UNTESTED) This repository servers as a template for creating React.JS - Django Web Applications. Note that the

Darryl See Wei Shen 5 Aug 19, 2022
Wrap the Blockchain API in Django!

django-blockchain Wrap the Blockchain API in Django. Installation pip install django-blockchain Add app in your settings.py INSTALLED_APPS = [ "d

Dmitry Kalinin 2 Feb 04, 2022
Organize Django settings into multiple files and directories. Easily override and modify settings. Use wildcards and optional settings files.

Organize Django settings into multiple files and directories. Easily override and modify settings. Use wildcards in settings file paths and mark setti

Nikita Sobolev 940 Jan 03, 2023
A simple trivia quizzz web app made using django

Trivia Quizzz A simple trivia quizzz web app made using django Demo http://triviaquizzz.herokuapp.com/ & https://triviaquiz.redcrypt.xyz Features Goog

Rachit Khurana 2 Feb 10, 2022
Atualizando o projeto APIs REST Django REST 2.0

APIs REST Django REST 3.0-KevinSoffa Atualização do projeto APIs REST Django REST 2.0-Kevin Soffa Melhorando e adicionando funcionalidades O que já fo

Kevin Soffa 2 Dec 13, 2022
Tutorial para o projeto negros.dev - A Essência do Django

Negros Dev Tutorial para o site negros.dev Este projeto foi feito com: Python 3.8.9 Django 3.1.8 Bootstrap 4.0 Como rodar o projeto? Clone esse reposi

Regis Santos 6 Aug 12, 2022
django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project. Inspired in the dashboard framework Dashing

django-dashing django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project.

talPor Solutions 703 Dec 22, 2022
Django/Jinja template indenter

DjHTML A pure-Python Django/Jinja template indenter without dependencies. DjHTML is a fully automatic template indenter that works with mixed HTML/CSS

Return to the Source 378 Jan 01, 2023
Hotwired/Turbo Django response helpers

This package provides helpers for server-side rendering of Hotwired/Turbo streams and frames. Disclaimer: the Hotwired/Turbo client libraries are, at

Hotwire for Django 66 Apr 07, 2022
A simple djagno music website.

Mrock A simple djagno music website. I used this template and I translated it to eng. Also some changes commited. My Live Domo : https://mrock.pythona

Hesam N 1 Nov 30, 2021
A feature flipper for Django

README Django Waffle is (yet another) feature flipper for Django. You can define the conditions for which a flag should be active, and use it in a num

950 Dec 26, 2022
A modern looking portfolio build with Django.

Django Portfolio A portfolio template using html/css/js in the frontend and Django as the backend framework. Cool features: smooth scrolling responsiv

1 Jan 19, 2022