Linux Programs

NFS
ELK
VirtualBox

NFS

NAS

Server

sudo apt-get update
sudo apt install nfs-common nfs-kernel-server portmap
sudo mkdir -p /data/tomcat
sudo chown nobody:nogroup /data/tomcat
sudo chmod 777 /data/tomcat

sudo vi /etc/exports

/data/tomcat *(rw,async,no_root_squash,no_subtree_check,no_root_squash)

sudo exportfs -a
sudo systemctl restart nfs-kernel-server

Client

sudo ufw allow from [clientIP or clientSubnetIP] to any port nfs
```
  sudo ufw allow from 192.168.56.1/24 to any port nfs
```
sudo ufw status
sudo apt-get update
sudo apt-get install nfs-common
sudo mkdir -p /data/tomcat
sudo mount 192.168.56.1:[nfs folder] [client folder]
```
 sudo mount 192.168.56.6:/data/tomcat /data/tomcat
```

fstab

 192.168.56.6:/data/tomcat /data/tomcat nfs rsize=8192,wsize=8192,timeo=14,intr

Backup

Host와 Guest의 공유 볼륨을 만든다. - 인터넷을 찾아서 동일하게 처리 https://helpdeskgeek.com/virtualization/virtualbox-share-folder-host-guest/
All you have to do is run your virtual machine and then click Devices and then Insert Guest Additions CD Image…

root로 실행하기

    vi /etc/cron.allow
    crontab -e 
    에디터를 변경하고 싶으면 select-editor

backup shell 만들기

export FinalBackupCnt=10
export FinalBackupDueDay=100
export BackupTarget=$1
export NOW=`date +%Y%m%d%H%m`
export FinalBackupDay=`date +%Y%m%d%H%m --date="$FinalBackupDueDay days ago"`
export BackupFolder=/host



# Backup Delete
DeleteBackupVm()
{

  echo "## Old $BackupTarget.tar backup file delete ! ##"
  TotalBackupCnt=`find $BackupFolder -name "*$BackupTarget.tar" | wc -l`
  echo ' >> Total backup files count = ' $TotalBackupCnt
  echo ' >> Final backup files remain count = ' $FinalBackupCnt
  TargetDeleteBackupCnt=`expr $TotalBackupCnt - $FinalBackupCnt`
  echo ' > Total backup files count - Final backup files count =' $TargetDeleteBackupCnt
  BackupAllFiles=$(find $BackupFolder -name "*$BackupTarget.tar" | awk -F "/" '{print $3}' | sort -n)


  cnt=1
  echo ' > ' $FinalBackupDueDay ' day before delete target snapshot list'
  for i in $BackupAllFiles
  do
    backupFileDate=`echo $i | awk -F "-" '{print $1}'`
    echo $cnt " : " $i
    if [ $backupFileDate -le $FinalBackupDay ]
    then
      echo "     DELETE!"
      if [ $TargetDeleteBackupCnt -ge $cnt ]
      then
           echo "     DELETEED! " $BackupFolder/$i
           rm $BackupFolder/$i
      else
           echo "     DELETE SKIP!"
      fi
    fi
  cnt=$(expr $cnt + 1)
  done
}


# Make Backup file
MakeBackUp()
{
  echo "Backup date is " $NOW
  tar cfP $BackupFolder/$NOW-$BackupTarget.tar /data/tomcat/$BackupTarget

}


echo "\n"
echo "=========== SNAPSHOT START ! ============="
echo ">> Backup Target : $BackupTarget"
echo ">> DATE : $NOW"
echo ">> Final Backup Due Day : $FinalBackupDueDay"
echo ">> Final Backup Day : $FinalBackupDay"

DeleteBackupVm
MakeBackUp

echo "============== $1 BACKUP END   ! ==========="
echo "\n"

Crontab -e 등록

0 4 2,9,16,23 * * /host/backup.sh meta >> /host/meta-backup.log
0 4 2,9,16,23 * * /host/backup.sh mysql >> /host/mysql-backup.log

ELK

옮김

Beats 6.4.2 : 가벼운 데이터 수집기. 로그, 메트릭, 네트워크, 이벤트, 감사데이터, 가동시간 모니터링 등을 실시간 수집
Logstash 6.4.2 : 데이터 집계, 변환, 저장. 다양한 소스에서 동시에 데이터를 수집하여 변환한 후 분석, 변환하여 저장소(elasticsearch)에 전달
Elasticsearch 6.4.2 : 분산형 RESTful 검색 및 분석 엔진. 정형, 비정형 데이터, 위치정보, 메트릭등 다양한 데이터를 원하는 방식으로 검색하고 저장
Kibana 6.4.2 : 로그 검색, 관계 분석 등을 위한 인터랙티브 시각화 도구

여기를 참조 : http://www.gnoti.com/index.php/2018/10/11/elk2/

Elasticsearch : https://medium.com/@ian.nam.kr/elasticsearch-%EC%84%A4%EC%B9%98-973fb438f2
- Download : wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.1-linux-x86_64.tar.gz
- start :
```
   nohup /apps/elasticsearch-7.4.1/bin/elasticsearch &
```
- stop :
```
    ps -ef | grep 'Elasticsearch' | awk '{print($2)}' | xargs kill -9
```
- max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
  - https://abc2080.tistory.com/entry/%EC%97%90%EB%9F%AC-1-max-virtual-memory-areas-vmmaxmapcount-65530-is-too-low-increase-to-at-least-262144
- the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
  - ```
  network.host: ["0.0.0.0"]
  
  cluster.initial_master_nodes: ["0.0.0.0"]
```
Kibana
- Download : wget https://artifacts.elastic.co/downloads/kibana/kibana-7.4.1-linux-x86_64.tar.gz
- config에서 server.host를 0.0.0.0으로 변경해야 외부에서 접근가능
Logstash
- Download : wget https://artifacts.elastic.co/downloads/logstash/logstash-7.4.1.tar.gz
- ```
./logstash-plugin update logstash-input-beats
```

filebeat

   https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.4.1-linux-x86_64.tar.gz

VirtualBox

가상머신 실행 하기
- /usr/bin/VBoxManage startvm Ubuntu-Server -type headless

vmdk --> vdi 사이즈 변경

schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ VBoxManage clonehd "/home/schmitt//Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstalled-disk1.vmdk" "/home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vid" --format VDI
Warning: program compiled against libxml 208 using older 207
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone hard disk created in format 'VDI'. UUID: 22ad08e4-737a-406e-af81-9401ce15c4ce
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ VBoxManage showhdinfo /home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vidWarning: program compiled against libxml 208 using older 207
UUID:           22ad08e4-737a-406e-af81-9401ce15c4ce
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vid
Storage format: VDI
Format variant: dynamic default
Capacity:       11125 MBytes
Size on disk:   9013 MBytes
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ 
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ 
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ VBoxManage modifyhd "/home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vid" --resize 51200
Warning: program compiled against libxml 208 using older 207
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$ VBoxManage showhdinfo /home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vidWarning: program compiled against libxml 208 using older 207
UUID:           22ad08e4-737a-406e-af81-9401ce15c4ce
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /home/schmitt/Desktop/data/after/VirtualBoxIMG/AcnDbServer001/Ubuntu1404OrgVersion_addinstall.vid
Storage format: VDI
Format variant: dynamic default
Capacity:       51200 MBytes
Size on disk:   9013 MBytes
schmitt@JCock:~/Desktop/data/after/VirtualBoxIMG/AcnDbServer001$

다음에 gparted를 실행하여 resize 또는 추가 볼륨을 만든다.

snapshotAllVM.sh

줄 번호 보이기/숨기기

   1 #!/bin/sh
   2 
   3 
   4 echo "\n\n"
   5 echo "##############################################"
   6 echo "############# MAKE SNAPSHOT START ! ##########" 
   7 echo "##############################################"
   8 
   9 
  10 echo "\n##  Virtural Images list  ##" 
  11 allVm=$(/usr/bin/VBoxManage list vms | awk '{print($1)}')
  12 for i in $allVm
  13 do
  14   vmname=$(echo $i | sed -e "s/\"//g")
  15   echo $vmname
  16   /home/smend/BackUpCenter/snapshotVM.sh $vmname
  17 done
  18 
  19 
  20 
  21 echo "\n\n"
  22 echo "##############################################"
  23 echo "############# MAKE SNAPSHOT END ! ############"
  24 echo "##############################################"
  25 echo "\n\n"

snapshotVM.sh

줄 번호 보이기/숨기기

   1 #!/bin/sh
   2 
   3 
   4 export _SDATE=$(date +"%Y%m%d%H")
   5 export FinalSnapshotCnt=10
   6 export FinalSnapshotDueDay=50
   7 export VMNAME=$1
   8 export NOW=`date +%Y%m%d%H`
   9 export FinalSnapshotDay=$((NOW -$(expr $FinalSnapshotDueDay*24)))
  10 
  11 
  12 echo "\n\n"
  13 echo "#########################################"
  14 echo "############# SNAPSHOT START ! ##########"
  15 echo "#########################################"
  16 echo ">> VMNAME : $1"
  17 echo ">> DATE : $_SDATE"
  18 echo ">> FinalBackupDay : $FinalSnapshotDueDay"
  19 
  20 echo "\n\n"
  21 echo "## Old $VMNAME snapshot delete ! ##"
  22 TotalsnapshotCnt=`VBoxManage snapshot $VMNAME list | grep Name | awk '{print ($2)}' | wc -l`
  23 echo ' >> Total snapshot count = ' $TotalsnapshotCnt
  24 echo ' >> Final snapshot remain count = ' $FinalSnapshotCnt
  25 TargetdeleteSnapshotCnt=`expr $TotalsnapshotCnt - $FinalSnapshotCnt`
  26 echo ' > Total snapshot count - Final snapshot count =' $TargetdeleteSnapshotCnt
  27 
  28 
  29 BackupAllFiles=$(VBoxManage snapshot $VMNAME list | grep Name | awk '{print ($2)}' | sort -n)
  30 cnt=1
  31 echo ' > ' $FinalSnapshotDay ' day before delete target snapshot list'
  32 for i in $BackupAllFiles
  33 do
  34   snapshotDate=`echo $i | awk -F "-" '{print $1}'`
  35   echo $cnt " : " $i 
  36   if [ $snapshotDate -le $FinalSnapshotDay ]
  37   then
  38           if [ $TargetdeleteSnapshotCnt -ge $cnt ]
  39           then
  40                 snapshotDate=`echo $i | awk -F "-" '{print $1}'`
  41                 #echo $snapshotDate " vs " $FinalSnapshotDay
  42                 if [ $snapshotDate -le $FinalSnapshotDay ]
  43                 then
  44                   echo "Delete snapshot $i"
  45                   VBoxManage snapshot $1 delete $i
  46                 fi
  47           else
  48                 echo "  >> skip snapshot : " $i
  49           fi
  50   fi
  51   cnt=$(expr $cnt + 1)
  52 done
  53 
  54 echo " >> Create snapshot !!" 
  55 VBoxManage snapshot $VMNAME take "$_SDATE-$VMNAME" --description "$_SDATE autometic snapshot from system "
  56 
  57 
  58 
  59 echo "\n\n"
  60 echo "#########################################"
  61 echo "############# SNAPSHOT END ! ############"
  62 echo "#########################################"
  63 echo "\n\n"