Managing files and directories in Hadoop Distributed File System (HDFS) is essential for any Hadoop user. Here's a quick cheat sheet of HDFS shell commands to help you navigate and manipulate your data efficiently:
mkdir: Create a new directory in HDFS.
bin/hadoop dfs -mkdir /datagen
Copy from Local: Copy a file from your local system to HDFS.
bin/hadoop dfs -copyFromLocal /home/ec2-user/data_10.txt /sample.txt
ls: List all directories in HDFS.
bin/hadoop dfs -ls /
lsr: Recursive version of the ls command, listing all directories and subdirectories.
bin/hadoop dfs -lsr /
Copy to Local: Copy a file from HDFS to your local system.
bin/hadoop dfs -copyToLocal /sample.txt /home/ec2-user/data_10.txt
cp: Copy files from a source location to a destination location in HDFS.
bin/hadoop dfs -cp /sample.txt /data_10.txt
mv: Move files from a source location to a destination location in HDFS.
bin/hadoop dfs -mv /sample.txt /sample2.txt
rm: Delete a file from HDFS.
bin/hadoop dfs -rm /sample.txt
rmr: Recursive version of the delete command, deleting directories and all their contents.
bin/hadoop dfs -rmr /datagen
tail: Display the last kilobyte of a file in HDFS.
bin/hadoop dfs -tail /sample.txt
These commands are invaluable for managing your HDFS files and directories effectively. Whether you're a beginner or an experienced Hadoop user, this cheat sheet will come in handy for your Hadoop data management tasks.
0 comments:
Post a Comment