/ Production Ready /

Acceleration FrameworkFor
For
Container Image Function Computation Vulnerability Scanning Dependency Packages Image Data Insight

A Dragonfly (CNCF incubating project) sub-project

Get Started

Provide Fast, Secure And Easy Access to Data Distribution Learn More

Nydus is a powerful opensource filesystem solution to form a high-efficiency image distribution system for Cloud Native workloads, such as container images, software packages, etc.

Nydus Core Design

Focus on Performance and Low Cost Read Doc
Performance
Second-level container startup speed, millisecond-level function computation code package loading speed.
Low Cost
Written in memory-safed Rust, numerous optimizations help improve memory, CPU, and network consumption.
Extensible and Flexible
Confidential container, vulnerability scan, support both runC native container and Kata Container.
Security
End to end data integrity check, Supply Chain Attack can be detected and avoided at runtime.
On-demand Load
Container images/packages are downloaded on-demand in chunk unit to boost startup.
Chunk Deduplication
Chunk level data de-duplication cross-layer or cross-image to reduce storage, transport, and memory cost.
End-to-end Data Integrity
File metadata and blob data can be both validated in transmission and at runtime.
Compatible with Ecosystem
Support with Registry, OSS, NAS, Shared Disk, and P2P service, compatible with the OCI spec.
Data Analyzable
Record accesses, data layout optimization, prefetch, IO amplification, abnormal behavior detection.

Nydus Ecosystem

Working Hard on Toolchain Support Contribute
Build
Nydusify
A CLI tool converts an OCI container image from source registry into a Nydus image.
Harbor Acceld
Provides a general service for Harbor to support image acceleration based on kinds of accelerator like Nydus and eStargz etc.
Buildkit
Provides the ability to build and export Nydus images directly from Dockerfile.
Ship
Dragonfly
Improve the runtime performance of Nydus image even further with the Dragonfly P2P data distribution system.
Storage Backend
Support for OCI-compatible distribution implementations such as Docker Hub, Harbor, Github GHCR, ACR, and Aliyun OSS-like object storage service.
Run
Nydus Snapshotter
Works as a containerd remote snapshotter to help setup container rootfs with Nydus image.
Nydus Docker Graphdriver
Works as a docker remote graph driver for preparing nydus image layer to bootstrap docker container.
KataContainers
Run Nydus image in Kata security container, Nydus has become a native image solution of Kata containers since v2.4.0.
For more details, see here.
EROFS
Nydus supports the Linux kernel enhanced read-only file system, which allows running Nydus image directly in-kernel for even greater performance improvement.

EROFS Read Doc

EROFS is a flexible, powerful, high-performance block-based modern read-only FS format.
EROFS is now deeply integrated with Nydus
Learn More
Uncompressed Seekable Metadata
Light-Weight Inode Metadata
Easy to Add On-Disk Payload
Data is Block-Addressed
Random Lookup Friendly On-Disk Directory Format
Tail-Packing Inline

Nydus Use Cases

Not Just Focus on Data Distribution
Container Image
Compared to OCI image, Nydus image pull times are reduced from minutes to seconds, in combination with Dragonfly P2P distribution, Nydus helps large-scale clusters save network bandwidth and reduce network load.
Function Computing
Reduces package download and decompression time, improves startup speed under high concurrency, supports high creation frequency and high deployment density, and reduces CPU and memory load. See usecase .
Vulnerability Scanning
Do a security scan of the image based on a list of known risks, thanks to the Nydus on-demand loading feature, which greatly increases the speed of scanning risky data, code, and packages.
Dependency Management
Use Nydus to increase the speed of dependency installation by reducing the network and IO load of the large number of small files generated by dependency package decompression. See usecase .
Data Insight
The implementation of Nydus at the file system level, can analyze the usage in image, file, and data chunk for business, achieve global data de-duplication, reduce the size of image layer for high-frequency incremental builds.

Nydus Community

Aliyun serverless image pull time drops from 20 seconds to 0.8s seconds.
Serving large-scale clusters with millions of container creations each day.
ByteDance
Serving container image acceleration in Technical Infrastructure of ByteDance.