A session where I got hive and hadoop to run
Jump to navigation
Jump to search
Mistakes and everything. Used https://phoenixnap.com/kb/install-hadoop-ubuntu and https://cwiki.apache.org/confluence/display/Hive/GettingStarted
1 ssh-keygen -t ed25519 -P ~/.ssh/id_ed25519 2 ssh-keygen -t ed25519 -P 3 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 4 cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys 5 chmod 0600 ~/.ssh/authorized_keys 6 ssh localhost 7 wget https://downloads.apache.org/hadoop/common/hadoop-3.2.1/hadoop-3.2.1.tar.gz 8 wget https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz 9 ls 10 cd 11 ls 12 wget https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz 13 ls 14 ls -lahtr 15 rm hadoop-3.3.1.tar.gz 16 wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz 17 ls 18 tar -xvf hadoop-3.3.1.tar.gz 19 ls 20 sudo vi .bashrc 21 vi .bashrc 22 pwd 23 vi .bashrc 24 ls 25 vi .bashrc 26 ls hadoop-3.3.1/lib/native/ 27 ls 28 source ~/.bashrc 29 vi ~/.bashrc 30 source ~/.bashrc 31 vi ~/.bashrc 32 source ~/.bashrc 33 sudo nano \$HADOOP_HOME/etc/hadoop/hadoop-env.sh 34 vi \$HADOOP_HOME/etc/hadoop/hadoop-env.sh 35 vi \$HADOOP_HOME/etc/hadoop/hadoop-env.sh 36 vi $HADOOP_HOME/etc/hadoop/hadoop-env.sh 37 ls /usr/lib/java 38 ls /usr/lib/jvm 39 ls /usr/lib/jvm/java-8-openjdk-amd64/ 40 vi $HADOOP_HOME/etc/hadoop/hadoop-env.sh 41 which javac 42 readlink -f /usr/bin/javac 43 source $HADOOP_HOME/etc/hadoop/hadoop-env.sh 44 vi hadoop-3.3.1/etc/hadoop/core-site.xml 45 ls /home/hadoop/ 46 mkdir tmpdata 47 vi hadoop-3.3.1/etc/hadoop/hdfs-site.xml 48 mkdir dfsdata 49 ls 50 mkdir dfsdata/namenode 51 mkdir dfsdata/datanode 52 vi hadoop-3.3.1/etc/hadoop/mapred-site.xml 53 vi hadoop-3.3.1/etc/hadoop/yarn-site.xml 54 hdfs 55 hdfs namenode -format 56 cd hadoop-3.3.1/sbin/ 57 ls 58 ./start-dfs.sh 59 ./start-yarn. 60 ./start-yarn.sh 61 jps 62 curl localhost:9870 63 curl -I localhost:9870 64 curl -I localhost:9870/index.html 65 curl localhost:9870/index.html 66 ss 67 man ss 68 ss -t -a 69 sudo ss -t -a 70 ss -t -a 71 ls 72 ./bin/beeline -u jdbc:hive2:// 73 ls 74 vi derby.log 75 hive 76 ./bin/hive 77 ./bin/hive --service metastore 78 ls 79 rm -r metastore_db/ 80 ./bin/schematool -initSchema -dbType derby 81 ./bin/hive 82 hdfs dfs -ls / 83 hdfs dfs -ls /user 84 hdfs dfs -ls /user/hive 85 hdfs dfs -ls /user/hive/warehouse 86 hdfs dfs -ls /user/hive/warehouse/pokes