In another of my medium article “Life of an Application Support Engineer” I talked about the followings:
- me
- career prospects
- interview
- first year
- second year
- promotion
- pitfalls
- tips to excel in application support
This article I will share some of the common technical tips for application support.
Table of Contents
- Regarding files
- SSH
- Networking
- Memory
- Storage
Regarding Files
Viewing files side-by-side in VIM
vim file1 file2 file3
:vsplit (for vertically splitting tabs), :split (for horizontally splitting tabs)
Typing :vsplit once will split window to two tabs
Typing :vsplit twice will split window to three tabs
Typing :vsplit thrice will split window to four tabs
Ctrl+W for switching between tabs
Click up down left right arrow key bunch of times to make sure it switched ok. This part could be buggy
:n for moving to the next file in the subtab, :N for moving to the previous file in the subtab
:h for reading help document, and can close by :q!
Side-by-side diff
sdiff
String searching for binary file
strings somebinaryfile | grep textuwanttofind
Read XML
xmllint — format myfile.xml | pygmentize -g | less -R -N
Tarball
- For viewing or listing
tar -tvf my-data.tar.gz
tar -tvf my-data.tar.gz ‘search-patter’
vim my-data.tar.gz
tar -xJf myfile.tar.xz — to-command=cat out.log | less (J for xz type files, x for extract, f for archive)
tar -axf .tar myfile -O | less
tar axf myfile.tar.xz out.log -O | less - Creating archive
tar cvpf myfile.tar *.log* — remove-files
tar -cvzf myfile.tar.gz file1 folder1 file2 - extracting
tar -xf archive.tar
Find hidden files
find . -name ‘.*’ -maxdepth 1 -ls;
Important linux adminitrative logs
/var/log/messages
Semaphores
They are used to access shared system resource such as shared memory. Gatekeepers to ensure that particular shared system resources are not accessible by multiple processes at the same time.
cat /proc/sys/kernel/sem
ipcs -ls
SSH
Bulk run commands in the background via ssh and for loop
for i in server1 server2 server3; do ssh $i ‘nohup sleep 90 > /dev/null 2>&1 &’; done
for i in server1 server2 server3; do ssh -t $i “hostname&&hostname | tail -1”; done
Run commands in SSH shell and stay in that shell
ssh -t username@server “ls -tlrh;bash -l”
-t: force pseudo-tty allocation, which is to execute arbitrary screen-based programs on remote machine.
tty: teletype terminals is a device implemented in software to input and output. It prints the file name of the terminal connected to standard input. e.g. /dev/pts/117
Networking
Check cname query
use dns function to find out
Check routing table
ip r
route
Check network interfaces
ip link show
lo — loopback interface
eth0 — first ethernet network interface
eth1 — second ethernet network interface
wlan0 — first wireless network interface
bond0 — aggregation of multiple parallel segments into one logical segment. Effectively, two interfaces are treated as one, and traffic is distributed across the component segments.
nmcli device status
nmcli connection show
netstat -i
ifconfig -a
/etc/sysconfig/network-scripts/ifcfg-bond0
Check open socket connections
netstat
-a = show both listening and non listening sockets
-n = show numeric network addresses
-p = show the name of program
networkinfo
Investgate packet drop and network statistics
ethtool -S eth0
NIC drop counters are logged to /var/log/counters/eth0
ifconfig bond0 | grep err
sar -n EDEV | egrep ‘bond0|IFACE’ | grep -v
ss
ifstat
Bandwidth usage and throughput test/file transfer
sudo iftop -P -i eth0
P = show port
B = convert bits to bytes
i = interface (wang ka)
第一行：带宽显示
中间部分：外部连接列表，即记录了哪些ip正在和本机的网络连接
中间部分右边：实时参数分别是该访问ip连接到本机2秒，10秒和40秒的平均流量
=>代表发送数据，<= 代表接收数据
底部三行：表示发送，接收和全部的流量
底部三行第二列：为你运行iftop到目前流量
底部三行第三列：为高峰值
底部三行第四列：为平均值
TX：发送流量
RX：接收流量
TOTAL：总流量
Cumm：运行iftop到目前时间的总流量
peak：流量峰值
rates：分别表示过去 2s 10s 40s 的平均流量
https://www.vpser.net/manage/iftop.html
iperf for file transfer in the network (like rsync or scp)
iperf -c (server address) *for connecting to a server
iperf -s *for creating a server
Memory
Look for VGROW (total virtual memory growth) in the output of atop to troubleshoot.
Storage
Check disk space and usage
df -h
du -h — max-depth=2 /tmp | egrep “M” | sort -rt ‘ ‘ -nk1 | head -10
du -h — max-depth=2 /tmp | egrep “G” | sort -rt ‘ ‘ -nk1 | head -10
List block devices (e.g. hard drive, RAM)
lsblk
show every mounted and unmounted devices
displays in a tree format with splitting partitions
mount (shows only the mounted partitions)
/dev/sda1 or /dev/sda2 or /dev/sda3 or /dev/sdb or /dev/sdc or /dev/sdd are for hard drive. Usually the SSD is mapped before the hard drive.
List of every possible device verbosely
lspci -vvv
NFS
lsof -N (this lists all the NFS files)
findmnt | grep nfs
/etc/exports : Its a main configuration file of NFS, all exported files and directories are defined in this file at the NFS Server end.
/etc/fstab : To mount a NFS directory on your system across the reboots, we need to make an entry in /etc/fstab.
/etc/sysconfig/nfs : Configuration file of NFS to control on which port rpc and other services are listening.
