In this tutorial we going to setup and configure NexentaEdge S3 compatible service.
NexentaEdge Extended S3 API provides unique benefits which can be useful for Machine Deep Learning, Big Data and IoT frameworks:
- Mount S3 objects for fast File/POSIX access avoid unnecessary copy, fetch only needed datasets
- Extended S3 feature set: Append, Range Writes, Object/Bucket snapshots, Key-Value Object access
- Data Reduction with global inline de-duplication, compression and erasure encoding
- Cost Reduction File/Block/DB access with S3 economics
Use cases details:
- Advanced Versioned S3 Object Append and RW "Object as File" access
- S3 Object as a Key-Value database, including integrations w/ Caffe, TensorFlow, Spark, Kafka, etc
- High-performance Versioned S3 Object Stream Session (RW), including FUSE library to mount an object
- Management API for Snapshots and Clones, including Bucket instantaneous snapshots
- Transparent NFS to/from S3 bucket access, “ingest via NFS, read via S3” or vice-versa
OPTIONAL: Prior to installation please ensure that you have ACTIVATION_KEY token available. If not yet register DevOps account, obtain license key here: https://community.nexenta.com/s/devops-edition
Give Edge-X S3 a try in easy to run 3 step procedure!
You now successfully setup and configured NexentaEdge cluster with S3 service!
Follow up with our Community!
Read more at https://github.com/Nexenta/edge-dev
Configuring NexentaEdge S3 Cluster
Start NexentaEdge data node container
NexentaEdge is designed to be "shared-nothing" scale-out SDS solution where data storage containers spread out across physical server nodes connected via Ethernet network (we call it Replicast/UDP backend network). In this first step we will setup first host node as Edge data node with a location where to keep data blobs:
In real deployments the above location ideally has to point to previously prepared mount point using ext4, xfs or zfs filesystems. More complex configuration allows usage of direct raw disk interface as well.
Now that we setup data blobs location, start nexenta/nedge daemon and Edge-X S3 compatible service
docker run --name s3data -v /etc/localtime:/etc/localtime:ro -v /var/tmp/data:/data -d nexenta/nedge start -j ccowserv -j ccowgws3
It will take few minutes for docker image to download. Please be patient..