The Anatomy of an Isolate Cloud
(And if you haven't, deploy a Fresh site today!)
Achieving this deployment speed was no mistake. Designing Deno Deploy was an opportunity to re-imagine how we want the developer experience of deploying an app to be, and we wanted to focus on speed and security. Instead of deploying VMs or containers, we decided on V8 isolates, which allows us to securely run untrusted code with significantly less overhead.
This blog post explains what V8 isolates are and outlines the major architectural pieces of the Deno Deploy isolate cloud.
Deno Deploy Projects and Deployments
Within Deno Deploy, users and organizations manage their applications as
projects. Deploy projects can be linked to GitHub repositories, enabling
code to be redeployed on each
git push. Projects also allow applications to
manage environment variables, provision TLS certificates, and associate domain
names. The Deploy dashboard also provides views of logs and analytics out of the
box for increased visibility into an application's behavior.
As previously mentioned, Deploy projects can be linked to GitHub repositories so that running applications are updated each time a change is pushed to the backing repository. In Deploy, each of these updates is known as a deployment. Deployments are immutable snapshots of an application. Each deployment has a unique URL that can be used to test or otherwise interact with the application over the Internet. A project can have many deployments, but only one of the deployments can be tagged as the production environment at any given time.
Figure 1 shows the Deployments dashboard for deno.land. Each deployment shows its unique deno.dev URL, as well as the git commit information that was used to create the deployment. The "Prod" label indicates the deployment that is currently running in production on deno.land.
Routing Requests to Deployments
But how do users' HTTP requests reach a running deployment in the Deno Deploy cloud?
Each deployment is exposed to the Internet via one or more public URLs. During DNS resolution, all of these URLs are mapped to Deno Deploy's public IPv4 or IPv6 address. This ensures that all deployment traffic is routed to the Deploy servers.
Depending on where the user is in the world, sending all traffic to the same destination can be very inefficient. For example, if a user is located in Australia, but all of the Deploy servers are located on the East coast of the United States, each HTTP request, including all necessary TCP handshaking, would need to travel thousands of miles around the globe.
Once an incoming request is routed to the appropriate edge location, it's handed off to a runner process. As its name implies, a runner is responsible for running applications, including receiving traffic and routing it to the correct deployment.
Deploy maintains a mapping table between domain names and deployments to serve
this purpose. Depending on the protocol in use, the HTTP/1.1
Host header or
:authority pseudo-header is used to determine the domain that the
request is intended for. Then, the TLS
Server Name Indication (SNI)
is used to determine which TLS certificate to use for the connection. Finally,
the runner checks the mapping table to determine the correct deployment to send
the request to. (As an aside, Deploy HTTP traffic is redirected to HTTPS, so the
SNI can be used reliably.)
The Runner as an Isolate Hypervisor
In a traditional cloud, the hypervisor is responsible for creating, running, monitoring, and destroying virtual machines. In Deploy, the runner acts as an isolate hypervisor, serving the same purpose, but working with processes running V8 isolates instead of virtual machines.
Isolate represents an isolated instance of the V8 engine. V8 isolates have completely separate states. Objects from one isolate must not be used in other isolates.
A high level architectural view of a runner process and deployment processes is shown in Figure 3.
Allowing many arbitrary users to upload and execute untrusted code is an extremely difficult technical challenge to solve effectively. Layered security is generally considered to be a good practice, but in this case it is a necessity because there is no single security mechanism that solves the Deno Deploy use case.
This section discusses some of the ways in which Deploy improves its security posture.
Running deployments in separate processes. V8 isolates are already... isolated... from one another. Deno Deploy goes a step further and enlists the operating system's help to improve the isolation of each deployment.
Hypervisor monitoring of resource utilization. The runner continuously tracks the metrics of all running deployments. This is necessary for billing purposes, but also allows the runner to enforce resource utilization quotas. Deployments that consume too many resources are terminated to prevent service degradation.
Restricting network access. Cloud providers and the operating system allow network access to be customized and locked down. Deploy also employs separate networks for internal control plane traffic and end user data plane traffic.
Restricting allowed system calls. As an added layer of security, Deploy uses seccomp filtering to limit the system calls that user code is allowed to execute.
Everyday, as we're iterating and improving on Deno Deploy, more developers and businesses choose to run production-ready applications and infrastructure with us. We're confident that our architectural decisions enable them to focus on building for their users, without having to worry about security or performance.
Check out Deno Deploy and let us know what you think!
Do you find these challenges interesting to solve? We're hiring!