Python Web Hosting Guide

Web hosting is a sub-module of LeanEngine that allows you to develop a web application with Python. It provides functions including cloud functions, hooks, hosting for static files, and custom routing, with your own domains bound to the service. You can use it to create a landing or download page for your mobile application, develop an admin console or a complete website, or run custom logic that has to be on the server side.

If you don't know how to create a LeanEngine project, debug it locally, or deploy it to the cloud, please first read LeanEngine Quick Start. Other related documents include:

• LeanEngine Overview (overview of all features)
• Command-Line Interface Guide
• LeanCache Guide (using the in-memory cache service to improve performance)

This guide uses Python as an example, but LeanEngine supports many other languages as well. You can choose the one you are familiar with for development:

• Node.js
• Python (current)
• PHP
• Java

Starting with the Example Project

Provided that you have [installed the command-line tool(leanengine_cli.html#installation), to create a new project from our example skelton, you just need one command:

lean init

Follow the prompts to enter the project information.

Before you launch a Python project locally, make sure the following dependencies are ready:

• python: Make sure the version you installed locally is the same as the one you want to use for production. This helps you avoid compatibility issues. We recommend that you use pyenv to manage your local Python versions.
• pip: Used to install third-party dependencies.
• virtualenv: Optional. We recommend that you use virtualenv or similar tools to create an independent Python environment for each project to avoid potential conflicts.

Please make sure the dependencies above are installed locally properly, then execute the following command under the project directory to install the third-party dependencies for the project:

pip install -r requirements.txt

Running and Debugging Locally

Once all dependencies are installed, you can use our command-line interface in the project root directory to start a development server:

lean up

For more information regarding command-line interface and local debugging, see Command-Line Interface Guide.

Deploying to the Cloud

Run the following command in your project root directory:

lean deploy

Setting up Second-Level Domains

In the "Web Hosting Domain" section of your app's Dashboard > LeanEngine > Settings, you can fill in a custom second-level domain. For example, if you set it to be myapp, you can access your website through http://myapp.avosapps.us.

Our second-level domains support HTTPS by default.

If no request has been made to your app within the past 30 days and you haven't purchased a standard instance, the second-level domain may be recycled.

Project Skeleton

Your project has to follow the following structure to be recognized by LeanEngine and operate properly.

LeanEngine Python runs your project following WSGI standards, so wsgi.py and requirements.txt must exist in the project root directory, with .python-version and runtime.txt to be optional files. When running your program on LeanEngine, wsgi.py will be loaded and the global variable application will be called as a WSGI function, so make sure application exists in your wsgi.py as a global variable, function, or class, and conforms WSGI standards.

For example, a minimal Flask application only contains two files (requirements.txt excluded and local debugging unsupported):

# app.py

from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
    return "hi"

# wsgi.py
from app import app
application = app

For more information about WSGI, please refer to PEP333.

Adding Third-Party Dependencies

You can fill in the third-party dependencies of the project in requirements.txt, with one module per line:

# From # sign to the end of the line are comments
leancloud>=2.9.1,<3.0.0
Flask>=1.0.0                                       # Specify version number/range of versions
git+https://github.com/foo/bar.git@master#egg=bar  # Use the remote addresses of version management tool such as Git/SVN

See pip 19.0.1 Documentation > User Guide > Requirements Files for details.

After deployment, LeanEngine will automatically install dependencies specified in requirements.txt. You can use the following command to install dependencies when running and debugging locally:

pip install -r requirements.txt

When deploying your project to the production environment, it is advisable to specify the exact versions (foo==1.0.0) of dependencies to avoid compatibility issues.

Specifying a Python Version

You can select the Python version used to run the code in a way similar to how pyenv does: put the desired version of Python in .python-version under the project root directory (like 3.6.1). When deploying your project to LeanEngine, the version specified here will be automatically installed.

If pyenv is used locally, it will refer to the same file for Python version. We recommend that you use pyenv for local development to ensure that the local environment is the same as the one on LeanEngine. See pyenv's GitHub repository for instructions on installation.

At this time, only CPython is supported. PyPy, Jython, IronPython, or other Python implementations are not supported. It is recommended that you use Python 3.5 or above. If you are still using Python 2, please use Python 2.7.

As mentioned previously, all the frameworks compatible with Python WSGI standard can be deployed to LeanEngine, including Flask, Django, and Tornado. We offer sample projects built with Flask and Django, which can be used as the template to start building your project:

• Flask
• Django

Using LeanStorage

In LeanEngine, you can use LeanStorage as the backend database, as well as other features provided by LeanCloud. LeanCloud Python SDK makes it easy for you to use these features.

Installation

Dependencies will be automatically installed upon deployment if leancloud is added into requirements.txt. When developing and debugging your project locally, you can run the following command under the project root directory to install dependencies:

pip install -r requirements.txt

Initialization

wsgi.py is the first file to be executed, so it is recommended that you initialize the LeanCloud Python SDK in this file:

import os
import leancloud

APP_ID = os.environ['LEANCLOUD_APP_ID']              # Obtain App ID from environment variable LEANCLOUD_APP_ID
APP_KEY = os.environ['LEANCLOUD_APP_KEY']            # Obtain App Key from environment variable LEANCLOUD_APP_KEY
MASTER_KEY = os.environ['LEANCLOUD_APP_MASTER_KEY']  # Obtain Master Key from environment variable LEANCLOUD_APP_MASTER_KEY

leancloud.init(APP_ID, app_key=APP_KEY, master_key=MASTER_KEY)
# Set this to be True if you need to access LeanCloud services with Master Key
leancloud.use_master_key(False)

Now you can use all LeanCloud Python SDK features in your project. See LeanStorage Python Guide for more information.

Health Monitoring

While your project is being deployed, LeanEngine's hypervisor will check if it is started successfully every second. The deployment is considered a failure if the check still fails after 30 seconds. Even after the project is deployed successfully, we still run health checks regularly to ensure that your app is running normally. If the check fails, the LeanEngine hypervisor will automatically restart your app.

The URLs for health monitoring include the index page of your project (/) and /__engine/1/ping which is handled by the Python SDK. As long as one of them returns the HTTP status code 200, the project is considered to be normal. So make sure your program uses the Python SDK or has its index page return 200 as the HTTP status code. Besides this, in order to support cloud functions and hooks in LeanEngine, the hypervisor will use the URL /1.1/functions/_ops/metadatas to communicate with the Python SDK. Please make sure to have this URL handled by the Python SDK or return 404 as the HTTP status code to indicate that cloud functions and hooks are not used.

See Using LeanStorage for how to install and use the LeanStorage Python SDK.

Session Management

Python SDK provides a WSGI middleware leancloud.engine.CookieSessionMiddleware that manages sessions of users (leancloud.User) with cookies. To use the middleware, replace the following line in wsgi.py:

application = engine

with:

application = leancloud.engine.CookieSessionMiddleware(engine, secret=YOUR_APP_SECRET)

You need to pass in a secret which is used to sign the cookie (required). The middleware will record the login status of leancloud.User in the cookie. When a user opens the application, the SDK will check if the user is logged in. You can obtain the logged-in user via leancloud.User.get_current().

Other options supported by leancloud.engine.CookieSessionMiddleware include:

• name: The name of the cookie (defaults to leancloud:session).
• excluded_paths: URL paths that do not trigger session token validations. For example, when handling static pages, you can skip validating session tokens, which improves efficiency.
• fetch_user: Whether or not to automatically fetch the currently logged in leancloud.User object (defaults to False). If it is set to be True, each HTTP request will initiate an API call to fetch the user object. If it is set to be False, you can only access the session_token attribute of the leancloud.User.get_current() by default. You can manually fetch the entire user when needed.
• expires: The expiration time of the cookie (see Werkzeug Documentation).
• max_age: The time before the cookie expires (see Werkzeug Documentation).

Implementing Common Features

Sending HTTP Requests

You can use any Python modules to send HTTP requests. The built-in urllib would work, but it would be more convenient if you use the third-party module requests. Actually, the LeanCloud Python SDK uses requests to invoke LeanCloud REST API interfaces. Thus if your project already depends on the LeanCloud Python SDK, you do not need to specify requests as a dependency of your project. Otherwise you can add a new line requests in requirements.txt and execute pip install -r requirements.txt to install it.

import requests

response = requests.post('http://www.example.com/create_post', json={
    'title': 'Some Amazing Title',
    'body': 'Some awesome content.',
})

print(response.json())

Obtaining Client IP

You can get the client's IP directly from the x-real-ip field in the HTTP header:

Flask:

from flask import Flask
from flask import request

app = Flask(__name__)

@app.route('/')
def index():
    print(request.headers['x-real-ip'])
    return 'ok'

Django:

According to Django's Documentation, the prefix HTTP_ will be added to third-party headers and - will be replaced by _. Therefore, you need to access the IP address with HTTP_X_REAL_IP.

def index(request):
    print(request.META['HTTP_X_REAL_IP'])
    return render(request, 'index.html', {})

For other frameworks, please refer to their documentation for instructions.

Uploading Files

Websites hosted on LeanEngine can handle file uploads with file-related APIs of LeanCloud Python SDK.

Assuming the frontend HTML code looks like this:

<form enctype="multipart/form-data" method="post" action="/upload">
  <input type="file" name="iconImage">
  <input type="submit" name="submit" value="submit">
</form>

Now define the handler function for uploading files, build a Form object, and parse req as a parameter. This will save the uploaded file into a temporary directory, and construct the files object:

Flask:

# app is your Flask instance

@app.route('/upload', methods=['POST'])
def upload():
    upload_file = request.files['iconImage']
    f = leancloud.File(upload_file.filename, data=upload_file.stream)
    print(f.url)
    return 'File uploaded successfully!'

For other frameworks, please refer to their documentation for instructions.

Sessions

It is recommended that you use the built-in session component of the web framework.

CSRF Tokens

If your project uses cookies as an authentication method (like the SDK's CookieSession middleware), then there is a risk of CSRF attack which allows other sites to forge malicious requests with valid cookies.

CSRF tokens can be used to defend such attacks. A CSRF token is a random string generated by the server and passed to the client (usually through cookie). When the client makes a request, it must include the same CSRF token in the request body or the header. The server then verifies if the CSRF token is valid before processing the request.

LeanCache

LeanCache is a cache service provided by LeanCloud for high-traffic sites.

To use LeanCache, add redis into requirements.txt as a dependency:

Flask>=0.10.1,<1.0.0
leancloud>=2.0.0,<3.0.0
…
redis>=2.10.5,<3.0.0

Then create a Redis connection with the following code:

import os
import redis

r = redis.from_url(os.environ.get("REDIS_URL_<instance_name>"))

For more instruction on how to use LeanCache, see LeanCache Guide.

Redirecting to HTTPS

For security consideration, it is recommended that all sites use HTTPS by default. LeanEngine supports such feature with a middleware that forces your website {App Domain}.avosapps.us to be accessed via HTTPS. Simply add the following code to have it enabled:

import leancloud

application = get_your_wsgi_func()

application = leancloud.HttpsRedirectMiddleware(application)

After your site is deployed to the production environment, all users accessing your website's second-level domain through HTTP will be redirected to HTTPS-enabled URLs.

Deploying and Publishing

Deploying with Command-Line Interface

Run the following command in your project root directory:

lean deploy

The command-line interface makes it easy to deploy projects, publish projects, view project status, and view logs. It allows you to deploy the same project to multiple applications as well. See Command-Line Interface Guide for more instructions.

Dependency Caching

LeanEngine implements a caching mechanism to speed up the build process (the process that installs dependencies for your project). For each new build, if the dependencies are not changed, the dependencies installed in the past will be reused and only the project code will be replaced.

The cached dependencies may become ineffective for various reasons, so it's not guaranteed that every build will make use of them.

If you are encountering problems with installing dependencies, you can enable the "Download latest dependencies" option when deploying on the web console, or add the --no-cache option when deploying with the command-line interface.

lean deploy --no-cache

Deploying from Git Repository

You can also deploy your project from a Git repository. LeanCloud doesn't offer version control services, so you need to push the project to a Git repository of your own choice. Good third-party websites that provide Git hosting services include GitHub, BitBucket, and GitLab.

After you push your code to a repository accessible from the Internet, please enter the URL of the Git repository (like https://github.com/<username>/<repoName>.git) into LeanEngine's settings page on the web console.

If the repository is private, you need to provide an SSH address (git@github.com:<username>/<repoName>.git) instead. You also need to provide the public key generated by LeanEngine as a deploy key to the Git service provider. For example, if you are using GitHub, you can add a deploy key by going to the repository's Settings > Deploy keys and click on Add deploy key.

After this, you will be able to deploy your project on the web console. It will deploy the code of the master branch by default, but you can specify a branch name, a label, or a specific commit when deploying.

If your repository contains submodules, LeanEngine will pull them automatically.

You can also set up a webhook so that each time you make a push to the Git repository, your project will be deployed from it. To do so, go to your app's Dashboard > LeanEngine > Deploy > Deploy from git > Auto deploy to obtain the deploy token and webhook URL. You can use the deploy token to construct HTTP requests to trigger deployments. Filling in the code repository branch and LeanEngine environment, a corresponding webhook URL will be generated automatically. The code from the specified branch will be deployed to the specified environment once the webhook URL receives any POST request.

For example, if you are using GitHub, you can go to your repository's Settings > Webhooks and add the URL as the Payload URL of a new webhook (leave other settings as default). By doing so, the designated branch will be deployed to LeanEngine each time you make a push to any existing branch. The reason we cannot perform a deployment based on the branch you are pushing to is that GitHub doesn't allow you to set up webhooks for specific branches, plus LeanEngine is not designing this feature for any specific platforms so it couldn't read the contents in a POST request. However, you can use GitHub Actions to control the timing of your deployment in a more granular way. Please refer to the example shown on the dashboard.

Staging Environment and Production Environment

By default, you application has a production environment for LeanEngine and the corresponding domain is {App Domain}.avosapps.us. There is a trial instance in the production environment to run your project.

When the trial instance in the production environment is upgraded to a standard instance, there will be a new staging environment with the corresponding domain stg-{App Domain}.avosapps.us, with both environments accessing the same LeanStorage data. You can test your code by deploying it to the staging environment each time you make a change and publish it to the production environment when ready. If you want to have a test environment with an independent data source, you can create a separate application.

Sometimes you may need to know the environment the current process is running in (development, staging, or production) and have your project behave differently:

import os

env = os.environ.get('LEANCLOUD_APP_ENV')
if env == 'development':
  # Development environment started by command-line interface
  do_some_thing()
elif env == 'production':
  # Production environment
  do_some_thing()
elif env == 'stage':
  # Staging environment
  do_some_thing()

In SDK, you can specify the environment where requests from clients are sent to:

[AVCloud setProductionMode:YES]; // Production environment (default)

[AVCloud setProductionMode:NO]; // Staging environment

AV.setProduction(true); // Production environment (default)

AV.setProduction(false); // Staging environment

leancloud.use_production(True) # Production environment (default)

leancloud.use_production(False) # Staging environment
# Call it before `leancloud.init`

LeanClient::useProduction(true); // Production environment (default)

LeanClient::useProduction(false); // Staging environment

AVCloud.setProductionMode(true); // Production environment (default)

AVCloud.setProductionMode(false); // Staging environment

Cloud Runtime Environment

System-Level Dependencies

You can specify system-level dependencies by changing systemDependencies of leanengine.yaml:

systemDependencies:
  - imagemagick

LeanEngine supports the following dependencies:

• ffmpeg: An audio and video processing tool and library.
• imagemagick: An image processing tool and library.
• phantomjs: A headless WebKit browser (deprecated).
• chrome-headless: A headless Chrome browser (takes more time to be deployed and consumes a lot of CPU and memory; if you use puppeter, you need to pass {executablePath: '/usr/bin/google-chrome', args: ['--no-sandbox', '--disable-setuid-sandbox']} to puppeteer.launch; does not support Java).
• node-canvas: System dependencies required for installing node-canvas (you still need to install node-canvas by yourself).
• python-talib: A library for performing technical analysis of financial market data.

Adding system dependencies will slow down the deployment. Please don't add unused dependencies.

Environment Variables

LeanEngine provides the following environment variables by default:

Variable Name	Description
LEANCLOUD_APP_ID	The App ID of the current application.
LEANCLOUD_APP_KEY	The App Key of the current application.
LEANCLOUD_APP_MASTER_KEY	The Master Key of the current application.
LEANCLOUD_APP_ENV	Current environment: Depending on how you start the local development instance, this variable may not exist or may be set to development. Staging environment: stage. Production environment: production.
LEANCLOUD_APP_PORT	The port listened by the project. Other ports are not accessible from the public network.
LEANCLOUD_API_SERVER	The URL used to access LeanStorage (looks like https://api.leancloud.cn). It is based on the location of the data center used to host the project and may change over time. You can use it when making requests to LeanStorage or other LeanCloud services with our REST API. Please do not use https://api.leancloud.cn directly in LeanEngine.
LEANCLOUD_AVAILABLE_CPUS	The number of CPUs available to the LeanEngine instance. It is based on the size of your instance and should be larger than 1 for large instances (like those with 2 CPUs and 1024 MB memory). Please do not use the number of CPUs of the operating system directly, or your project may be terminated if excessive threads are opened up.
LEANCLOUD_APP_GROUP	The group this instance belongs to. When using the LeanEngine Group Management feature, this becomes the name of the group.
LEANCLOUD_REGION	The region where the instance is located at. For LeanCloud International version, the value is US.
LEANCLOUD_VERSION_TAG	The instance deployment version.

You can also add custom environment variables in your app's Dashboard > LeanEngine > Settings > Custom environment variables. Variable names can only contain letters, numbers, and underscores, and can only begin with a letter. The value must be a string. The updated environment variables will take effect after the next deployment.

A common practice of using environment variables is to store configurations with them. By doing so, the behavior of your project can be changed by updating environment variables only without touching the code. You can also store secrets keys of third-party services as environment variables so that they are not exposed in the code.

# Use custom environment variables in LeanEngine Python Environment
import os

MY_CUSTOM_VARIABLE = os.environ.get('MY_CUSTOM_VARIABLE')
print(MY_CUSTOM_VARIABLE)

By default, the built-in and custom environment variables can only be read by the application at runtime. If you want to read these environment variables while installing dependencies or compiling, add the following line to leanengine.yaml:

exposeEnvironmentsOnBuild: true

Load Balancing

On LeanEngine, requests from the clients go through a load balancer before they reach to your app. The load balancer handles SSL encryption and decryption, data compression, etc., so you don't have to add HTTPS- or gzip-related functions to your app. Since the load balancer handles SSL encryption and decryption, the request URLs received by the web framework always start with http. You need to check the X-Forwarded-Proto HTTP header to detect if the original requests use HTTP or HTTPS.

The load balancer also enforces limitations that the maximum size of each request is 100 MB (including uploading files to LeanEngine), the maximum time for processing each request is 60 seconds, and a WebSocket will be disconnected if no data goes through in the past 60 seconds.

File System

You can write temporary files into /home/leanengine or /tmp, up to 1 GB in total.

Each time a project is deployed to LeanEngine, a new container will be created. Even though you don't initiate deployments, the system may also occasionally perform clean-ups. Therefore, you can't treat the local file system as a persistent storage.

If the files written are relatively large, we recommend that you delete the files after using them. If more than 1 GB of disk space is used, you may receive "Disk quota exceeded" errors. In this situation, you can redeploy your project to clear out the files.

Logs

You can view LeanEngine's deployment and runtime logs at Dashboard > LeanEngine > App logs. The logs can be filtered by levels.

You can also export the logs of the past seven days to local files with the command-line interface.

Logs are written to either "Standard Output" or "Standard Error", corresponding to the log levels info and error. For example, the following code will write 'Hello!' at the info level:

Python 2:

import sys

print 'Hello!' # info
print >> sys.stderr, 'An error occurred!' # error

Python 3:

import sys

print('Hello!') # info
print('An error occurred!', file=sys.stderr) # error

LeanEngine's access logs can be exported at Dashboard > LeanEngine > Access logs.

Time Zones

The time zone used on the server is UTC.

Public IP Addresses

If you need to configure IP whitelists on third-party services and need to obtain LeanEngine's public IP addresses, please go to Dashboard > LeanEngine > Settings > Public IP addresses.

We will try our best to avoid changing public IP addresses, but cannot eliminate the possibility of making changes. In the case of IP-related problems, please double check if the public IP addresses are changed.

Custom Domains

We recommend that you register your own domain and bind it to your LeanEngine production environment. You can bind a custom domain at Account settings > Domain binding.