AWS

AWS Log EC2 to S3 .sh

KK_Ryong 2024. 10. 28. 16:24

#! /bin/sh

BASE_DT=`date +%Y%m%d -d -1days`
SOURCE_DIR=/app/server/was-tomcat-9.0.83/log/
S3_BUCKET=hm-shopby-logs
S3_PATH=s3://$S3_BUCKET/service-log/year=${BASE_DT:0:4}/month=${BASE_DT:4:2}/day=${BASE_DT:6:2}
UPLOAD_LOG=/app/aws-cli/util/log/service.log.$BASE_DT
INST_ID=`ec2-metadata -i | awk -F' ' '{print $2}'`

S3_FILE_LIST=()
S3_LS_RESULT=`/app/aws-cli/bin/aws s3 ls $S3_PATH/$INST_ID --profile happy | awk -F' ' '{print $4}' | awk -F'.' '{print "service.log."$2".log"}'`
for s3 in $S3_LS_RESULT; do
        S3_FILE_LIST+=($s3)
done

cd $SOURCE_DIR

S3_LS_RESULT=`ls service.log.$BASE_DT*`
echo "[START]`date '+%Y-%m-%d %H:%M:%S'`" >> $UPLOAD_LOG
for i in $S3_LS_RESULT; do
        IS_EQUAL=0

        for s3 in ${S3_FILE_LIST[@]}; do
                if [[ "$i" == "$s3" ]]; then
                        IS_EQUAL=1
                        break
                fi
        done

        if [[ $IS_EQUAL == 1 ]]; then
                continue
        fi

        NEW_FILE=$INST_ID'-'$i
        /usr/bin/cp $i $NEW_FILE
        /usr/bin/tar cvzf ${NEW_FILE%.*}.tar.gz $NEW_FILE
        /app/aws-cli/bin/aws s3 cp ${NEW_FILE%.*}.tar.gz $S3_PATH/${NEW_FILE%.*}.tar.gz --profile happy 2>> $UPLOAD_LOG

        rm -rf $NEW_FILE
        rm -rf ${NEW_FILE%.*}.tar.gz
done
echo "[FINISH]`date '+%Y-%m-%d %H:%M:%S'`" >> $UPLOAD_LOG

'AWS' 카테고리의 다른 글

AWS Log EC2 to S3 .sh (이전 30일 치)  (0) 2024.11.01
S3 비용 최적화  (0) 2024.07.19
Redis용 Amazon MemoryDB 구성  (0) 2024.03.07
AWS EC2 blue/green Deploy (신규 서비스 생성)  (0) 2023.10.12
ECS blue/green | jenkins  (0) 2023.09.18