Have you heard of computing on traditional disk-based or flash-based technologies? We all use disks/flash storage in our laptops, desktop or servers sitting in the data center.
What is Apache Ignite In-Memory Data Fabric ?
It's high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time "orders of magnitude faster than possible with traditional disk-based or flash-based technologies."
Fabric: In information technology, fabric is a synonym for the words framework or platform.
You can view Ignite as a collection of independent, well-integrated, in-memory components geared to improve performance and scalability of your application.Source
How Ignite fits with HDFS? (Ignite file system , IGFS)
IGFS shakes hand with HDFS. Hadoop can run over IGFS in plug-n-play fashion and significantly reduce I/O and improve both, latency and throughput.
Why do we need another layer on the top of HDFS?
IGFS supports dual-mode. As you can see in the above pic, it can be deployed as main file system or it can sit on top of HDFS to provide caching layer, provides highly configurable read-through and write-through behaviour.
IGFS serve as an in-memory caching layer over disk-based HDFS.
Installation scroll down to "In-Memory Hadoop Accelerator:"
The best part of the install is HADOOP_README.txt and useful documentation. Check out HDP and Ignite setup guide
