/ Production Ready /

Acceleration FrameworkFor
For
Container Image Function Computation Vulnerability Scanning Dependency Packages Image Data Insight

A sub-project of Dragonfly (CNCF incubating)

Provide Fast, Secure And Easy Access to Data Distribution Learn More

Nydus is a powerful opensource filesystem solution to form a high-efficiency image distribution system for Cloud Native workloads, such as container images, software packages, etc.

Nydus Core Design

Focus on Performance and Low Cost Read Doc
Performance
Second-level container startup speed, millisecond-level function computation code package loading speed.
Low Cost
Written in memory-safed language Rust, numerous optimizations help improve memory, CPU, and network consumption.
Flexible
Supports container runtimes such as runC, Kata, CRI-O and confidential containers.
Security
End to end data integrity check, Supply Chain Attack can be detected and avoided at runtime. And also support vulnerability scanning capabilities.
On-demand Load
Container images/packages are downloaded on-demand in chunk unit to boost startup.
Chunk Deduplication
Chunk level data de-duplication cross-layer or cross-image to reduce storage, transport, and memory cost.
POSIX Compatibility
In-Kernel EROFS, FUSE and VirtioFS support provides full POSIX compatibility.
Compatible with Ecosystem
Support with Registry, OSS, NAS, Shared Disk, and P2P services. And compatible with the OCI specification.
Data Analyzable
Record accesses, data layout optimization, prefetch, IO amplification, abnormal behavior detection.

Nydus Ecosystem

Working Hard on Toolchain Support Contribute
Build
Nydusify
A CLI tool converts an OCI container image from source registry into a Nydus image.
Harbor Acceld
Provides a general service for Harbor to support image acceleration based on kinds of accelerator like Nydus and eStargz etc.
Buildkit
Provides the ability to build and export Nydus images directly from Dockerfile.
Nerdctl
A docker like CLI to convert/run Nydus image in containerd.
Ship
Dragonfly
Improve the runtime performance of Nydus image even further with the Dragonfly P2P data distribution system.
Harbor
Supports Nydus image distribution and preheat with Dragonfly.
Storage Backend
Supports for OCI-compatible distributions and object storage services, (Such as Docker Hub, Harbor, Github GHCR, ACR, Aliyun OSS, AWS S3, NAS, Local Disk).
Run
Containerd (Nerdctl run) / Kubernetes
Works with Containerd / kubernetes to run Nydus image.
Docker / Moby
Run Nydus image in Docker / Moby container with Containerd and nydus-snapshotter.
KataContainers
Run Nydus image in Kata security container, Nydus has become a native image solution of Kata containers since v2.4.0.
For more details, see here.
CRI-O / Podman
An additional layer storage plugin provided Nydus images lazy pulling ability for CRI-O/Podman.
EROFS
Nydus supports the Linux kernel enhanced read-only file system, which allows running Nydus image directly in-kernel for even greater performance improvement.

EROFS Read Doc

EROFS is a flexible, powerful, high-performance block-based modern read-only FS format.
EROFS is now deeply integrated with Nydus
Learn More
Uncompressed Seekable Metadata
Light-Weight Inode Metadata
Easy to Add On-Disk Payload
Data is Block-Addressed
Random Lookup Friendly On-Disk Directory Format
Tail-Packing Inline

Nydus Use Cases

Not Just Focus on Data Distribution
Container Image
Compared to OCI image, Nydus image pull times are reduced from minutes to seconds, in combination with Dragonfly P2P distribution, Nydus helps large-scale clusters save network bandwidth and reduce network load.
Function Computing
Reduces package download and decompression time, improves startup speed under high concurrency, supports high creation frequency and high deployment density, and reduces CPU and memory load. See usecase .
Vulnerability Scanning
Do a security scan of the image based on a list of known risks, thanks to the Nydus on-demand loading feature, which greatly increases the speed of scanning risky data, code, and packages.
Dependency Management
Use Nydus to increase the speed of dependency installation by reducing the network and IO load of the large number of small files generated by dependency package decompression. See usecase .
Data Insight
The implementation of Nydus at the file system level, can analyze the usage in image, file, and data chunk for business, achieve global data de-duplication, reduce the size of image layer for high-frequency incremental builds.

Nydus Community

Partners & Adopters
Aliyun serverless image pull time drops from 20 seconds to 0.8s seconds.
Serving large-scale clusters with millions of container creations each day.
Serving container image acceleration in Technical Infrastructure of ByteDance.
Starting to deploy millions of containers with Dragonfly and Nydus.
The startup time of micro service has been greatly improved, and reduced the network consumption.
Dramatically reduce the pull time of container image which embedded machine learning models.
Open Source Communities