AWS & Hadoop (Flume) (컴퓨터 윈도우의 포트 포워딩 방법 netsh)
AWS & Hadoop (Flume) (컴퓨터 윈도우의 포트 포워딩 방법 netsh)wget https://dlcdn.apache.org/flume/1.9.0/apache-flume-1.9.0-bin.tar.gzroot @ip-172-31-34-166:/opt/flume/conf# cp flume-env.sh .template flume-env.shroot@ip-172-31-34~166:/opt/flume/conf# cp flume_conf.properties.template flume_conf.propertieswget https://dlcdn.apache.org/flume/1.9.0/apache-flume-1.9.0-bin.tar.gzroot @ip-172-31-34-166:/opt/flume/conf# cp flume-env.sh .template flume-env.shroot@ip-172-31-34~166:/opt/flume/conf# cp flume_conf.properties.template flume_conf.properties/opt/flume/conf/flume-env.sh파일작성 JAVA을 내보냅니다_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64export FLUME_HOME=/opt/flumeexport PATH=$PATH:다음과 같이 입력합니다:$FLUME_HOME/bin/opt/flume/conf/log4j.properties파일수정 flume.log.dir=./logs이걸 flume.log.dir=/opt/flume.logs이렇게 수정[1] 로그 수집 agent(EC2)[1] 로그 수집 agent(EC2)#에이전트설정 agent_transfer.channels=channel1 agent_transfer.sources=readeragent_transfer.sinks=avro-forward-sink#channel설정 agent_transfer.channels.channel1.type=memoryagent_transfer.channels.channel1.capacity=1000agent_transfer.channels.channel1.transaction능력=100#소스 설정 agent_transfer.sources.reader.type=exec.agent_transfer.sources.reader.command=tail-F/logs/apache/access.log.`/bin/date+%Y%m%d`agent_transfer.sources.reader.command=/usr/bin/tail-F/var/log/apache2/access.log#실시간으로로그를 띄우는명령어 agent_transfer.sources.reader.logStdErr=trueagent_transfer.sources.reader.restart=true#sinks설정 agent_transfer.sinks.avro-forward-sink.type=avro.avro로 전달한다. agent_transfer.sinks.avro-forward-sink.hostname=125.133.75.24#보내려고하는 서버의 IPagent_transfer.sinks.avro-forward-sink.port=33156#포트포워딩을위해서 […]
AWS & Hadoop (Flume) (컴퓨터 윈도우의 포트 포워딩 방법 netsh) Read More »