The assignment
The Broken Phone Project is a modern dance film and online platform, also known as the movement archive, where people can upload their own interperation of dance routine. Every new uploaded video will automatically be added to a chain of already uploaded videos in order to create a never ending video. Together with Lockhorst Software Solutions Studio Terabyte built a full stack solution for The Broken Phone Projet, consisting of a data pipeline which process video data, a server which handles the communication between the front-end and the video pipeline and a front-end which fits the modern style of the project.
Video data pipeline
The first big challenge was building the video data pipeline. At it's core it needed to fullfill a number of functions:
- Converting the uploaded videos to a standard file format, video size and video quality;
- Editing the videos together;
- Storing all individual videos and the main video in one central place;
- Updating the status of every video in the SQL database.
Besides meeting the requirements above, each video also had to be aproved by an admin first. This to ensure that no random videos could make it in the main video. This resulted in the following flow:
For the editing and rendering of the videos FFmpeg has been chosen. This is one of the most used multimedia frameworks for working with audio and video programatically. It doesn't just offer all necessary options, such as dynamically adjust the dimensions of the videos, but is also reliable and fast.
Videos are saved with the help of Amazon s3: Amazon's cloud solution for saving large files such as photos and videos. Amazon s3 is a good choice for this project due to both it being cost effective and user friendly.
Studio Terabyte and Lockhorst Software Solutions used Python to bring FFmpeg and Amazon s3 together into one fully automated pipeline.
Server architecture
After completing the video data pipeline the next step was the building of the server which the pipeline could be intergrated with. Here Python was chosen again, specifically the Flask webframework. Flask, one of the most widley used webframeworks due to its minimalistic design and flexibility to build the functionality necessary, is a good choice for a custom solution.
A different wish from The Broken Phone Project was to be able to save information about the uploaded videos, such as admin approval status or information if a video is already part of the main video. Sqlite3 is being used to save this information: a lightweight SQL database which is ideal for application which don't need to write or read a database continuously.
The final part of the server architecture was the integration of the video data pipeline itself. For this integration Redis has been used. This is a "key-value" database which lives directly in the memory of the server (RAM) instead of on a harddrive. When the server sends a command to the video pipeline, for example to change the size of an uploaded video, the necessary information is saved in Redis and automatically detectred by the video pipeline. This happen asynchronously which means the end user does not need to wait until the server is finished editing the video.
Front-end
Studio Terabyte and Lockhorst Software Solutions also designed the front-end of The Broken Phone project. The website is built as a platform where the never ending dancefilm can be seen, information about the project can be found and visitors can upload their own interpertation of the danceroutine. The design is minimal, modern and focuses on the content. Different elements of the websites are dynamic and will be updated automatically, such as when a nieuw video is uploaded.
To built the front-end Flask's templating engine Jinja2 was used. This templating engine makes it possible to build dynamic pages which are directly integrated with the server. For the styling the popular CSS framwork Boostrap, which makes it easy to build mobile friendly websites which look good on any screen size, was used.
Deployment
After all the parts for The Broken Phone Project have been built and tested, it was time to deploy everything. Studio Terabyte and Lockhorst Software Solutions chose a Digital Ocean server with extra CPU power to make the video editing process as efficient as possible and to ensure that it will not affect other parts of the project, such as the front-end.
Results
- An efficient fully automated video editing pipeline.
- A reliable and fast server.
- A dynamic front-end with a modern design.
- 86 uploaded videos since launch.
- Visitors from more than 25 different countries.