The Need for a New Approach to Fast I/O

New non-volatile memory (NVM) device technologies are proliferating and media capacities are increasing rapidly. Diverse many-core processor strategies are pushing higher I/O volumes and more demanding I/O profiles. Traditional file systems only crudely manage flash at scale whereas HDD performance degrades as concurrency increases. IME introduces a new layer, above the file system that:

• Maximizes flash media performance and longevity.
• Improves the efficiency of network and storage through adaptive I/O and data coalescing.
• Efficiently feeds the diverse I/O requirements of modern workloads and mixed CPU/GPU environments.
• Delivers the data rates needed across commercial analytics in machine learning.

Software-Only or Optimized Appliance

IME is available as software only or as an integrated appliance. For software only deployments contact your DDN representative for the latest information on qualified third party components and configurations.

The IME240 is built on a best-of-breed 2U commodity storage server that delivers over 20GB/s of bandwidth and over 1M IOPs direct to applications. IME240 has up to 23 NVMe SSD drives for data and flexible N+K erasure coding.

IME is well suited for a variety of use cases including:

• Machine Learning and Analytics: Massive, flash cache with over 600K read IOPs in 2U and 20GB/s to support large scale datasets at speed.
• Application Ensembles: Multiple, simultaneous applications use cache to support communication throughout the job.
• Application Workflows: Cache stores a common dataset used by a succession of independent applications.
• In-situ Analysis: Real-time output of an application’s data in cache analyzed in-situ.
• Application Pre-Processing: Pre-processing job places output in cache, ready for main run.
• Application Post-Processing: Main run outputs to cache, post-processing commences in-situ.
• Checkpointing: Stores checkpoints faster than a parallel file system and keeps them in cache to support restarts.