Getting Started (on AWS)¶
OpenShot Cloud API is easy to install and configure. Launch your own private instance using AWS Marketplace, or by following this link: https://aws.amazon.com/marketplace/pp/B074H87FSJ/. The following video walks you though the configuration process.
This tutorial video covers the basics of how to configure an OpenShot Cloud API instance, and walks you through the process of creating your first video project.
Launch an Instance¶
After you launch an instance of the OpenShot Cloud API, it is initially configured to run as both a web server and task server (server), and as a video processor (worker). Once you have started your instance, you can access the web server without changing anything.
Access your new instance in a web browser. Use its public DNS or IP address. http://YourInstanceIP/
Architecture¶
This diagram illustrates the basic architecture of OpenShot Cloud API. A server listens for video editing tasks, and then sends those to an available worker. A worker processes each job (one at a time), rendering a final video. When a worker is done, it can optionally POST to a webhook URL, upload the video to S3, or the video file can be downloaded as needed. All servers and workers share files using HTTP, for accessing source assets (i.e. logos, videos, music files) and for sharing the final rendered output video.
The server should be configured with a large enough EBS volume to handle all files you intend to store. And if you are done with a project after exporting/rendering, it is best to delete it (and the source files will be removed automatically).
REST: Representational State Transfer¶
REST is a design pattern for web services modeled around HTTP requests. Each operation uses its own HTTP method:
Method |
Description |
|---|---|
GET |
Get data from the API (i.e. request a project, clip, or export) |
POST |
Create data on the API (i.e. create a project, clip, or export) |
PUT |
Update existing data from the API (i.e. update a project, clip, or export) |
PATCH |
Update partial existing data from the API (i.e. update only a Clip’s position) |
DELETE |
Delete existing data from the API (i.e. delete a project, clip, or export) |
Before You Begin¶
OpenShot Cloud API uses AWS IAM user credentials to access S3 and SQS. Before you begin, you will need to create a new IAM user on AWS, and give it the following permissions: AmazonS3FullAccess & AmazonSQSFullAccess.
- The following details will be required:
- IAM User
Access Key
Secret Access Key
Permissions: AmazonS3FullAccess & AmazonSQSFullAccess
- SQS Queue
Create a queue named OpenShotAPIExportQueue
Standard queue with default settings
OpenShot Cloud API will not work without this configuration, so please verify you have things configured correctly before moving forward.
Server Settings¶
To configure the settings of your new instance, SSH into the instance using your private key and the ubuntu user:
ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx
Run the following command to configure your server for the first time:
config-openshot-cloud
When setting up your first instance, it is recommended to configure it as both a server
and worker, by choosing the first option: (B)oth.
Choose a ROLE for this instance.
(B)oth - Runs both the HTTP API, DB, and video processing
(S)erver - Runs only the HTTP API and the DB (no video processing)
(W)orker - Runs only the video processing tasks
When configuring your server, it is very important to provide the following AWS settings. If these settings are invalid, OpenShot Cloud API will fail to process video exports, and S3 features will throw errors.
# This information is available for your AWS IAM User (mentioned above)
AWS Access Key ID?
AWS Secret Access Key?
AWS SQS Queue? (OpenShotAPIExportQueue)
AWS Region (us-east-1, us-west-2, eu-west-1, etc...)?
Worker Settings¶
If you are only running a single instance (server and worker), you can skip this section. If you are launching additional worker instances, this section covers how to connect them back to the OpenShot Cloud API server.
To configure the settings of your new worker instance, SSH into the instance using your private key and the ubuntu user:
ssh -i private-key.pem ubuntu@xxx.xxx.xxx.xxx
Run the following command to configure your worker for the first time, and choose the second option: (W)orker.
config-openshot-cloud
For each worker instance you launch, you will be prompted to provide the Cloud API URL of the server (which can be an internal IP address) and to provide AWS Settings (for SQS and S3 - same as configuring the server above). This allows the worker to watch the SQS queue for new video export tasks/jobs, download and upload files from S3, and use the API URL to update the export record with progress, status, and the final rendered video file.
NOTE: The worker must be able to communicate over port 80 with the API server.
Now that you have configured your worker instance, it should be listening for new tasks from the server, and updating the Export record on the server when it completes tasks. Each Export record contains the hostname of the worker that processed it. So, create a test Project and then create an Export, and you will see the hostname of the worker added to the Export record.
Verify AWS Configuration¶
When config-openshot-cloud has completed, it will try and verify your AWS configuration, and will let you know if anything went wrong. If you see a failure, please double check your AWS credentials and SQS queue name and run config-openshot-cloud again. You can also access http://YourInstanceIP/aws/validate/ at any time to verify your AWS configuration.
Admin Interface¶
A full admin interface is available at http://YourInstanceIP/cloud-admin/, which allows you to create additional user logins, manage your video projects, and search your data.
Python Example Script¶
Here is a simple example client script, using Python 3. It connects, creates a new project, uploads a file, creates a clip, and then exports a new video. It should give you a good sense of how things work in general, regardless of programming language. Learn more about the API Endpoints.
import os
import time
from requests import get, post
from requests.auth import HTTPBasicAuth
PATH = os.path.dirname(os.path.realpath(__file__))
CLOUD_URL = 'http://cloud.openshot.org'
CLOUD_AUTH = HTTPBasicAuth('demo-cloud', 'demo-password')
##########################################################
# Get list of projects
end_point = '/projects/'
r = get(CLOUD_URL + end_point, auth=CLOUD_AUTH)
print(r.json())
##########################################################
# Create new project
end_point = '/projects/'
project_data = {
"name": "API Project",
"width": 1920,
"height": 1080,
"fps_num": 30,
"fps_den": 1,
"sample_rate": 44100,
"channels": 2,
"channel_layout": 3,
"json": "{}",
}
r = post(CLOUD_URL + end_point, data=project_data, auth=CLOUD_AUTH)
print(r.json())
project_id = r.json().get("id")
project_url = r.json().get("url")
##########################################################
# Upload file to project
end_point = '/projects/%s/files/' % project_id
source_path = os.path.join(PATH, "example-video.mp4")
source_name = os.path.split(source_path)[1]
file_data = {
"media": None,
"project": project_url,
"json": "{}"
}
r = post(CLOUD_URL + end_point, data=file_data, files={"media": (source_name, open(source_path, "rb"))}, auth=CLOUD_AUTH)
file_url = r.json().get("url")
print(r.json())
##########################################################
# Create a clip for the previously uploaded file
end_point = '/projects/%s/clips/' % project_id
clip_data = {
"file": file_url,
"position": 0.0,
"start": 0.0,
"end": 30.0,
"layer": 1,
"project": project_url,
"json": "{}"
}
r = post(CLOUD_URL + end_point, data=clip_data, auth=CLOUD_AUTH)
print(r.json())
##########################################################
# Create export for final rendered video
end_point = '/projects/%s/exports/' % project_id
export_data = {
"video_format": "mp4",
"video_codec": "libx264",
"video_bitrate": 8000000,
"audio_codec": "ac3",
"audio_bitrate": 1920000,
"start_frame": 1,
"end_frame": None,
"project": project_url,
"json": "{}"
}
r = post(CLOUD_URL + end_point, data=export_data, auth=CLOUD_AUTH)
export_url = r.json().get("url")
print(r.json())
##########################################################
# Wait for Export to finish (give up after around 40 minutes)
export_output_url = None
is_exported = False
countdown = 500
while not is_exported and countdown > 1:
r = get(export_url, auth=CLOUD_AUTH)
print(r.json())
is_exported = float(r.json().get("progress", 0.0)) == 100.0
countdown -= 1
time.sleep(5.0)
# Get final rendered url
r = get(export_url, auth=CLOUD_AUTH)
export_output_url = r.json().get("output")
print(r.json())
print("Export Successfully Completed: %s!" % export_output_url)
Node.js Example Script¶
Here is a simple example client script, using JavaScript and Node.js. It connects, creates a new project, uploads some files, creates some clips, exports a video, and then downloads the new video file. It should give you a good sense of how things work using asynchronous JavaScript. Learn more about the API Endpoints.
var request = require('request-promise');
var fs = require('fs');
var Promise = require("promise");
var promisePoller = require('promise-poller').default;
var projectId = '';
var projectUrl = '';
var exportId = '';
const projectData = {
'json': '{}',
'name': 'My Project Name'
};
const protocol = 'http';
const server = 'cloud.openshot.org';
const auth = { 'user': 'demo-cloud',
'pass': 'demo-password'};
// Create a new project
post('/projects/', projectData)
.then(function(response){
projectUrl = JSON.parse(response).url;
projectId = JSON.parse(response).id;
console.log('Successfully created project: ' + projectUrl);
// Add a couple clips (to be asynchronously processed)
const promiseArray = [];
promiseArray.push(
createClip({
"path": "media-files/MyVideo.mp4",
"position": 0.0,
"end": 10.0
}));
promiseArray.push(
createClip({
"path": "media-files/MyAudio.mp4",
"position": 0.0,
"end": 10.0
}));
promiseArray.push(
createClip({
"path": "media-files/Watermark.JPG",
"position": 0.0,
"end": 10.0,
"layer": 2
}));
// Wait until all files and clips are uploaded
Promise.all(promiseArray).then(function(responseArray){
// Export as a new video
exportData = {
"export_type": "video",
"video_format": "mp4",
"video_codec": "libx264",
"video_bitrate": 8000000,
"audio_codec": "ac3",
"audio_bitrate": 1920000,
"project": projectUrl,
"json": '{}'
};
post('/exports/', exportData)
.then(function(response){
// Export has been created and will begin processing soon
var exportUrl = JSON.parse(response).url;
exportId = JSON.parse(response).id;
console.log('Successfully created export: ' + exportUrl);
// Poll until the export has finished
var poller = promisePoller({
taskFn: isExportCompleted,
interval: 1000,
retries: 60*60*1000,
timeout: 2000
}).then(function(exportOutputUrl) {
// New exported video is ready for download now
console.log('Download ' + exportOutputUrl);
request(exportOutputUrl).pipe(fs.createWriteStream('Output-' + projectId + '.mp4'));
});
});
});
});
function isExportCompleted() {
return new Promise(function (resolve, error) {
get('/exports/' + exportId + '/', {})
.then(function(response) {
var exportStatus = JSON.parse(response).status;
var exportOutputUrl = JSON.parse(response).output;
if (exportStatus == 'completed') {
console.log('Export completed: ' + JSON.stringify(response));
resolve(exportOutputUrl);
}
});
});
}
function createClip(clip) {
// Create new File object (and upload file from filesystem)
var fileData = {
'json': '{}',
'project': projectUrl,
'media': fs.createReadStream(clip.path)
};
return new Promise(function (resolve) {
post('/files/', fileData)
.then(function(response) {
// File uploaded and object created
var fileUrl = JSON.parse(response).url;
console.log('Successfully created file: ' + fileUrl);
// Now we need to add a clip which references this new File object
var clipData = {
'file': fileUrl,
'json': '{}',
'position': clip.position || 0.0,
'start': clip.start || 0.0,
'end': clip.end || JSON.parse(response).json.duration,
'layer': clip.layer || 0,
'project': projectUrl
};
return post('/clips/', clipData).then(function(response){
var clipUrl = JSON.parse(response).url;
console.log('Successfully created clip: ' + clipUrl);
resolve();
});
});
});
}
function post(endpoint, data) {
// Prepare request object and POST data to OpenShot API
const r = request.post(protocol + '://' + auth.user + ':' + auth.pass + '@' + server + endpoint, function (err, response, body) {
if (err) {
return err;
} else {
console.log(response.statusCode + ': ' + body);
return body;
}
});
// Append form data to request form-data
const form_data = r.form();
for ( var key in data ) {
form_data.append(key, data[key]);
}
return r;
}
function get(endpoint, data) {
// Prepare request object and GET data to OpenShot API
return request.get(protocol + '://' + auth.user + ':' + auth.pass + '@' + server + endpoint, function (err, response, body) {
if (err) {
return err;
} else {
console.log(response.statusCode + ': ' + body);
return body;
}
});
}
C# Example Script¶
Here is a simple example client script, using C# (.NET version: 5.0.103). It connects, creates a new project, uploads a file, creates a clips, exports a video, and then prints the new video file link. It should give you a good sense of how things work in C# (This code has been made synchronous on purpose). Learn more about the API Endpoints. This also uses 2 NuGet packages: Newtonsoft, MimeMapping
using System;
using System.Collections.Generic;
using System.Threading;
using System.Net.Http;
using System.Net.Http.Headers;
using System.IO;
using Newtonsoft.Json.Linq;
using MimeMapping;
namespace OpenShot
{
class Program
{
static void Main(string[] args)
{
String MakeGetRequest(string url, HttpClient client)
{
Console.WriteLine("GET: " + url);
var response = client.GetAsync(url).GetAwaiter().GetResult();
Console.WriteLine("Response StatusCode: " + (int)response.StatusCode);
if (response.IsSuccessStatusCode){
return response.Content.ReadAsStringAsync().GetAwaiter().GetResult();;
}else{
Console.WriteLine("Request Failed with message:" +
response.Content.ReadAsStringAsync().GetAwaiter().GetResult());
}
return "";
}
String MakePostRequest(string url, Dictionary<string, string> formData, HttpClient client)
{
var requestContent = new FormUrlEncodedContent(formData);
Console.WriteLine("POST: " + url);
var response = client.PostAsync(url, requestContent).GetAwaiter().GetResult();
Console.WriteLine("Response StatusCode: " + (int)response.StatusCode);
if (response.IsSuccessStatusCode){
return response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
}else{
Console.WriteLine("Request Failed with message:" +
response.Content.ReadAsStringAsync().GetAwaiter().GetResult());
}
return "";
}
String MakePostRequestWithFile(string url, Dictionary<string, string> formData, HttpClient client, FileInfo fileInfo)
{
var requestContent = new MultipartFormDataContent();
var fileContent = new StreamContent(fileInfo.OpenRead());
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
Name = "\"media\"",
FileName = "\"" + fileInfo.Name + "\""
};
fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse(MimeMapping.MimeUtility.GetMimeMapping(fileInfo.Name));
foreach (KeyValuePair<string, string> kvp in formData)
{
requestContent.Add(new StringContent(kvp.Value), kvp.Key);
}
requestContent.Add(fileContent);
Console.WriteLine("POST: " + url);
var response = client.PostAsync(url, requestContent).GetAwaiter().GetResult();
Console.WriteLine("Response StatusCode: " + (int)response.StatusCode);
if (response.IsSuccessStatusCode){
return response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
}else{
Console.WriteLine("Request Failed with message:" +
response.Content.ReadAsStringAsync().GetAwaiter().GetResult());
}
return "";
}
HttpClient client = new HttpClient();
// Connection info
string serverUrl = "http://cloud.openshot.org/";
// Basic Auth
String username = "demo-cloud";
String password = "demo-password";
String encoded = System.Convert.ToBase64String(
System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(
username + ":" + password));
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", encoded);
////////////////////////////////////////////////////////////////////////////////////////
// Get list of projects
Console.WriteLine("\n Get list of projects \n");
String responseGetRequest = MakeGetRequest(serverUrl + "projects/", client);
Console.WriteLine(responseGetRequest);
////////////////////////////////////////////////////////////////////////////////////////
// Create new project
Console.WriteLine("\n Create new project \n");
Dictionary<string, string> payload = new Dictionary<string, string>
{
{ "name", "API Project" },
{ "width", "1920" },
{"height", "1080"},
{"fps_num", "30"},
{"fps_den", "1"},
{"sample_rate", "44100"},
{"channels", "2"},
{"channel_layout", "3"},
{"json", "{}" }
};
String responsePostRequest = MakePostRequest(serverUrl + "projects/", payload, client);
Console.WriteLine(responsePostRequest);
dynamic jsonObj = JObject.Parse(responsePostRequest);
Int32 projectId = jsonObj.id;
String projectUrl = jsonObj.url;
Console.WriteLine("Project ID: " + projectId);
Console.WriteLine("Project URL: "+ projectUrl);
////////////////////////////////////////////////////////////////////////////////////////
// Upload file to project
Console.WriteLine("\n Upload file to project \n");
payload = new Dictionary<string, string>
{
{"media", ""},
{"project", projectUrl},
{"json", "{}" }
};
String filePath = "D:\\test-file.mp4";
responsePostRequest = MakePostRequestWithFile(
projectUrl + "files/",
payload,
client,
new FileInfo(filePath)
);
Console.WriteLine(responsePostRequest);
jsonObj = JObject.Parse(responsePostRequest);
String fileUrl = jsonObj.url;
Console.WriteLine("File URL:" + fileUrl);
////////////////////////////////////////////////////////////////////////////////////////
// Create a clip for the previously uploaded file
Console.WriteLine("\n Create a clip for the previously uploaded file \n");
payload = new Dictionary<string, string>
{
{"file", fileUrl},
{"position", "0.0"},
{"start", "0.0"},
{"end", "2.0"},
{"layer", "1"},
{"project", projectUrl},
{"json", "{}"}
};
responsePostRequest = MakePostRequest(projectUrl + "clips/", payload, client);
Console.WriteLine(responsePostRequest);
////////////////////////////////////////////////////////////////////////////////////////
// Create export for final rendered video
Console.WriteLine("\n Create export for final rendered video \n");
payload = new Dictionary<string, string>
{
{"video_format", "mp4"},
{"video_codec", "libx264"},
{"video_bitrate", "8000000"},
{"audio_codec", "ac3"},
{"audio_bitrate", "1920000"},
{"start_frame", "1"},
{"end_frame", ""},
{"project", projectUrl},
{"json", "{}"}
};
responsePostRequest = MakePostRequest(projectUrl + "exports/", payload, client);
Console.WriteLine(responsePostRequest);
jsonObj = JObject.Parse(responsePostRequest);
String exportUrl = jsonObj.url;
Boolean isExported = false;
Int16 countdown = 500;
while (!isExported && countdown > 1){
String responseExportGetRequest = MakeGetRequest(exportUrl, client);
jsonObj = JObject.Parse(responseExportGetRequest);
String progress = jsonObj.progress;
if (progress != null && progress !=""){
float progressFloat = float.Parse(progress);
if (progressFloat == 100.0){
isExported = true;
}
}
countdown -= 1;
Thread.Sleep(5000);
}
responseGetRequest = MakeGetRequest(exportUrl, client);
jsonObj = JObject.Parse(responseGetRequest);
Console.WriteLine(responseGetRequest);
Console.WriteLine("Export Successfully Completed: " + jsonObj.output);
}
}
}