hadoop 2.6.4 window 10 pseudo distribution mode 설치

1. hadoop 2.6.4 binary 다운로드
   http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.6.4/hadoop-2.6.4.tar.gz
2. winutils 다운로드
   https://github.com/steveloughran/winutils 

   winutils-master.zip

   다운로드

3. eclipse hadoop 2.x plugin 다운로드
   https://github.com/winghc/hadoop2x-eclipse-plugin
4. eclipse > dorpins에 hadoop-eclipse-plugin-2.6.0.jar 복사
5. HADOOP_HOME 환경변수 설정 ( E:/hadoop-2.6.4  )
6. hadoop-2.6.4.tar.gz 압축해제 후 %HADOOP_HOME% 에 복사
7. winutils-master.zip 압축해제 후 hadoop-2.6.4/bin 파일을 %HADOOP_HOME%/bin에 복사
8. 디렉토리 생성
   %HADOOP_HOME%/data/namenode,
   %HADOOP_HOME%/data/datanode,
   %HADOOP_HOME%/temp,
   %HADOOP_HOME%/userlog
9. %HADOOP_HOME%/etc/hadoop 환경설정
   core-site.xml
   

   <configuration>
       <property>
           <name>hadoop.tmp.dir</name>
           <value>\hadoop-2.6.4\temp\</value>
       </property>
       <property>
           <name>fs.defaultFS</name>
           <value>hdfs://0.0.0.0:50071</value>
       </property>
   </configuration>

   hdfs-site.xml

   <configuration>
       <property>
           <name>dfs.replication</name>
           <value>1</value>
       </property>
       <property>
           <name>dfs.namenode.name.dir</name>
           <value>file:/hadoop-2.6.4/data/namenode</value>
           <final>true</final>
       </property>
       <property>
           <name>dfs.datanode.data.dir</name>
           <value>file:/hadoop-2.6.4/data/datanode</value>
           <final>true</final>
       </property>
   </configuration>

   mapred-site.xml
   

   <configuration>
       <property>
           <name>mapreduce.framework.name</name>
           <value>yarn</value>
       </property>
       <property>
           <name>mapred.job.tracker</name>
           <value>0.0.0.0:9001</value>
       </property>
       <property>
           <name>mapreduce.application.classpath</name>
           <value>
               /hadoop-2.6.4/share/hadoop/mapreduce/*,
               /hadoop-2.6.4/share/hadoop/mapreduce/lib/*,
               /hadoop-2.6.4/share/hadoop/common/*,
               /hadoop-2.6.4/share/hadoop/common/lib/*,
               /hadoop-2.6.4/share/hadoop/yarn/*,
               /hadoop-2.6.4/share/hadoop/yarn/lib/*,
               /hadoop-2.6.4/share/hadoop/hdfs/*,
               /hadoop-2.6.4/share/hadoop/hdfs/lib/*
           </value>
       </property>
   </configuration>

   yarn-site.xml
   

   <configuration>
       <property>
           <name>yarn.nodemanager.aux-services</name>
           <value>mapreduce_shuffle</value>
       </property>
       <property>
           <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
           <value>org.apache.hadoop.mapred.ShuffleHandler</value>
       </property>
       <property>
           <name>yarn.nodemanager.log-dirs</name>
           <value>\hadoop-2.6.4\userlog</value>
           <final>true</final>
       </property>
       <property>
           <name>yarn.nodemanager.local-dirs</name>
           <value>\hadoop-2.6.4\temp\nm-localdir</value>
       </property>
       <property>
           <name>yarn.nodemanager.delete.debug-delay-sec</name>
           <value>600</value>
       </property>
       <property>
           <name>yarn.application.classpath</name>
           <value>
               /hadoop-2.6.4/,
               /hadoop-2.6.4/share/hadoop/common/*,
               /hadoop-2.6.4/share/hadoop/common/lib/*,
               /hadoop-2.6.4/share/hadoop/hdfs/*,
               /hadoop-2.6.4/share/hadoop/hdfs/lib/*,
               /hadoop-2.6.4/share/hadoop/mapreduce/*,
               /hadoop-2.6.4/share/hadoop/mapreduce/lib/*,
               /hadoop-2.6.4/share/hadoop/yarn/*,
               /hadoop-2.6.4/share/hadoop/yarn/lib/*
           </value>
       </property>
   </configuration>

   hadoop-env.cmd
   

   set JAVA_HOME=E:\hadoop-2.6.4\jre
   set HADOOP_HOME=E:\hadoop-2.6.4
   set PATH=%PATH%;%JAVA_HOME%\bin;E:\hadoop-2.6.4\bin;E:\hadoop-2.6.4\sbin;

10. java 1.8.x jre %HADOOP_HOME% 복사
11. 관리자모드 콘솔에서 %HADOOP_HOME%/bin이동 ,hdfs namenode -format
12. %HADOOP_HOME%\sbin 이동 start-dfs.cmd , start-yarn.cmd 실행
13. Eclipse Run On Hadoop 실행시
    Run Configuration → Classpath → User Entries → Advanced Options → Add External Folder → %HADOOP_HOME%/etc/hadoop
    job.setJar("jar full path");

※심볼릭 링크 관련 오류시 gpedit.msc → 컴퓨터구성 → Windows 설정 → 보안 설정 → 로컬 정책 → 사용자 권한 할당 → 심볼 링크 만들기 → 로그인유저 추가