Tips and Tricks

1. SSH break-out:

   <enter>~.
   <enter>~?
   

2. Shadowed window screenshot in mac:

   command-shift-4 then hit <spacebar>, then click in the window
   

3. selecting a path when double clicking in Terminal.app in mac:

   command-shift-doubleclick
   

4. Copy

   rsync -ave ssh ~/dir/ 192.168.16.131:dir/
   rsync -ave ssh HOSTNAME:/dir /dir/
   

5. Show hidden files in Finder:

   defaults write com.apple.finder AppleShowAllFiles TRUE
   killall Finder
   (to disable set to FALSE)
   

6. Show space-available at bottom of Finder windows:

   View > Show Status Bar
   

7. Bash loops

   % for h in foobar-{app1,app2,lb1,lb2,utility1}.baz; do ssh $h 'echo `hostname -f` `readlink /usr/local | cut -d/ -f4`'; done
   % i=0 ; while [ $i -ne 50 ]; do i=$(($i+1)); echo -n "=== $i ===" ; time curl http://assets1.grouponcdn.com/static/stylesheets/app/home/index-iFTURNG0.css -s -o /dev/null; done
   % for h in splunk{1,2,3,4,5,6,7}; do ssh $h 'hostname -f && df -h'; done
   % for h in 192.168.40.{70,71,84,85}; do echo -n "$h:"; ipmitool -I lanplus -H $h -U ADMIN -P ADMIN mc info; done
   % while : ; do echo foo ; sleep 5; done
   % sh -c 'while : ; do sh -c "date ; time curl -s http://config/host/HOSTNAME -o /dev/null ; date" >> curl.1.log 2>&1 ; sleep 5 ; done'
   

8. cut use

   bin/add_package lockrun-2011.09.02_15.52 `grep lockrun hostclasses/*.yml | cut -d: -f1 | xargs echo`
   vim `git stat | grep modified | cut -d: -f 2`
   

9. Package/hostclass manipulation:

   date +%Y.%m.%d_%H.%M
   git stat hostclasses | grep modified | cut -d: -f2 | cut -d/ -f2 | cut -d. -f1 | awk '{print echo "git tag " $1 "-2011.09.02_16.48"}'
   

10. Finding non-ascii characters:

    find app  | xargs grep -P "[\x80-\xFF]" | awk -F':' '{print $1}' | sort | uniq -c
    

11. mv with shell expansion with {}

    0 eweathers@G12885(~) % mv .rvm{,.bak}
    0 eweathers@G12885(~) % echo .rvm{,.bak}
    .rvm.bak
    0 eweathers@G12885(~) % echo .rvm{.bak,123,1,1}
    .rvm.bak .rvm123 .rvm1 .rvm1
    
    for i in `ls -l /usr/bin/* | grep Ruby.framework | awk '{print $9}'`; do echo sudo mv `dirname $i`/`basename $i @`{,.bak}; done
    

12. Using diff+ssh in a single line to compare across hosts:

    diff <(ssh HOSTNAME cat /etc/ssh/sshd_config) <(ssh HOSTNAME2 cat /etc/ssh/sshd_config)
    

    That trick is called "process substitution". Here are some related tips: http://www.linuxtutorialblog.com/post/tutorial-the-best-tips-tricks-for-bash http://www.commandlinefu.com/commands/using/diff

    https://gist.github.com/erikdw/0670fe94fa6027fbf1ebb6a44efab99c

13. tcpdumps: (no -s 65535 needed on recent tcpdumps)

http://danielmiessler.com/study/tcpdump/

(a) find dns requests (adjusted for being on hosts in terremark): sudo tcpdump -i eth1 -c3000 -l -n dst port 53 | grep api (thepoint@tm22-s00311)

 sudo tcpdump -i eth0 -c3000 -l -n port 3133

(b) capture all data:

 sudo tcpdump -i eth0 -w /tmp/1.pcap

(c) capture all data except port 22 (SSH):

 sudo tcpdump -i eth0 -w /tmp/1.pcap port not 22

(d) capture just dns:

 sudo tcpdump -i eth0 -w /tmp/1.pcap port 53

(e) find nfs requests

 sudo tcpdump -n -i any -c10000 dst port nfs | grep --line-buffered '"' | cut '-d"' -f2

(f) print HTTP GET requests

 sudo tcpdump -A -i eth0 -vvv -s 500 'tcp port 80 and ip[2:2] > 40 and tcp[tcpflags] & tcp-push != 0 and dst port 80' -f

(g) debugging ganglia's gmetad

 sudo tcpdump -w /tmp/gmetad.pcap -s 0 -vxli lo src host localhost and dst host localhost and dst portrange 8660-8777

(h) capturing traffic for a particular source subnet

 sudo tcpdump -i eth0 -w /tmp/1.pcap src net 50.115.209.0/24

(i) capturing traffic for a particular host with a particular port

 sudo tcpdump -i eth0 -w /tmp/1.pcap host 10.20.43.21 and port 22

(j) capture multicast or broadcast

 sudo tcpdump -i eth0 -w /tmp/1.pcap -n "multicast or broadcast"

(k) show outbound TCP traffic that is not SSH

 sudo tcpdump -i eth0 tcp and src host `ip address show dev eth0 primary | sed -n 's,^    inet ,,p' | sed 's,/.*$,,'` and port not 22

(l) rotating output file every 600 secs, retain only 18 files

 sudo tcpdump -w avatax-slow-pings-%Y%m%d%H%M%S.dump -G 600 -W 18 -vv '(host avatax.avalara.net and port 443)'

(j) tcpdump -i eth0 -w /tmp/1.pcap 'tcp port 8080 or tcp portrange 31000-31100'

14. SSHing thru a bastion to another host using 2 different keys: ssh -i ~/.ssh/us_east.pem -o ProxyCommand='ssh bastion.east-aws.domainname nc %h %p' root@SOMETARGETHOST

    • The pem key here is for sshing into the SOMETARGETHOST host as root.
    • My default ssh key is being used for executing the proxy command of sshing into the bastion.east-aws.domainname

15. Find out which processes are listening on TCP ports on Mac: sudo lsof -P -i tcp | grep -i listen

16. Create SSH tunnel to a particular port on a server (TARGETHOST) thru a bastion (ssh jump) host (BASTIONHOST): To use localhost:8001 to connect to port 80 of TARGETHOST ssh -L8001:BASTIONHOST:80 -qnN BASTIONHOST

To use localhost:2200 to connect to port 22 of TARGETHOST ssh -L2200:BASTIONHOST:22 -qnN BASTIONHOST

Another version: ssh -f -N -L 50000:TARGETHOSTSHORTNAME:50000 BASTIONHOST

17. Create a general HTTP proxy over an SSH tunnel thru a bastion:

ssh -L 21785:127.0.0.1:21785 -R 21785:127.0.0.1:21786 -L3133:TARGETHOST:3128 BASTIONHOST -qnN

Then you set your HTTP proxy (in your app's config) to localhost:3133, and send your HTTP requests thru it, and they will be proxied out thru the final host specified (TARGETHOST in the case above).

TODO: study the options here and grok them. NOTE: this cmd is created automatically using the autossh package and this cmd:

autossh -M "20001" -f '-L3131:TARGETHOST:3128 BASTIONHOST' -qnN

18. Translate Excel CSV to have unix-parseable newlines instead of ^M chars:

tr -s '^M' '\n' < foo-types.csv > foo-types-newlines.csv

(where ^M is entered as "ctrl-V-M")

Alternatively:

tr -s '\015' '\012'

19. History expansion in zsh http://www.acm.uiuc.edu/workshops/zsh/history/hist_expn.html

!^ - first arg !$ - last arg

20. Printing last N commands in zsh, and getting the time spent in each: fc -D -l -1000

21. Printing all commands in zsh history: fc -l 1

22. Suppress command numbers in zsh history: fc -ln 1

23. add prefix to input lines coming in, and combine into a single line with the strings separated by spaces: some-cmd | perl -pe '$_ = "hostclasses/" . $_' | tr -s '\012' ' '

24. Fix broken ssh-agent forwarding that is blocking a deploy. Try in a fresh shell:

eval ssh-agent ssh-add

(re-run your deploy command)

25. Ruby 1-liner to print a range of numbers in a comma sequence ruby -e '(51..75).each { |n| print "#{n}," }' Better (no trailing comma): ruby -e 'puts (51..75).inject{|s,n| "#{s},#{n}"}'

works: for h in 192.168.64.{51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75}; do echo -n "$h:"; ipmitool -I lanplus -H $h -U ADMIN -P ADMIN power status; done does not work: for h in 192.168.64.{ruby -e 'puts (51..75).inject{|s,n| "#{s},#{n}"}'}; do echo -n "$h:"; ipmitool -I lanplus -H $h -U ADMIN -P ADMIN power status; done

26. Slow-running tail of large file across NFS to keep filehandle open: thepoint@tm50-s00123 ~ $ perl -e 'open my $fh, "/shared39/thepoint/sitemap.xml" or die; while(<$fh>) { print $_; select undef, undef, undef, 0.5 } close $fh'

For the record, here's the change he applied on both tm50-s00039 (NFS primary) and tm27-s00202 (warm backup):

27. See entire history of changes for a file: git log --follow -p

28. Debugging "rake test" tasks: a) use rdebug: http://www.offbyzero.com/resources/debugging_rails_rdebug b) print messages: $stderr.puts c) can run the test directly; e.g., ruby test/colo_test.rb d) enable verbose output: rake test TESTOPTS="-v"

29. git: Show entirety of a file at a particular change (hash):

git show :

30. Generate SSH thumbprint from private key: ssh-keygen -lf

31. If you notice inconsistencies with order of output of "ls" cmds for files on different hosts, it is likely due to inconsistent locale settings.

32. If a username is longer than 8 characters, then "ps aux" will substitute the uid for username. Explanation for why "ps aux" shows a UID instead of the name touch_deploy

33. If linux is hanging after the "SELinux: Disabled at runtime" prompt, you probably have bad entries in /etc/fstab: http://www.spamstopshere.com/blog/2008/07/21/linux-hangs-at-selinux-disabled-at-runtime/

34. Connecting to someone's screen session in readonly mode:

35. Executing bash aliases in remote SSH cmd:

bash -ic

http://stackoverflow.com/questions/1198378/ssh-command-execution-doesnt-consider-bashrc-bash-login-ssh-rc

for h in {foo,bar,baz}; do echo -n "$h: " ; ssh -T $h bash -ic hostclass; done

36. Screwed up terminal (e.g., missing cursor, cat'ed a binary file and screen is full of gibberish):

reset

38. Delete deferred mail from postfix (e.g., on our nagios hosts)

[root@nagios1 log]# postsuper -d ALL postsuper: Deleted: 22718 messages

39. Restart network interface in gentoo:

sudo /etc/init.d/net.eth1 restart

40. Checking your current "run level" in linux:

/sbin/runlevel who -r

41. Reverse the lines in a file

tail -r

42. Running sar

sar -q -f /var/log/sa/sa25

(figure out which file from the file timestamp)

43. Escape shell meta characters (from my-vip-flip on bzr)

Escape shell meta-characters

for(my $i = 0; $i &lt;= $#args; $i++) { $args[$i] =~ s|([^\w/:-])|\$1|g; }

44. top w/ resident memory in bytes: top -o res -b

45. disable/enable crontabs for current user:

VISUAL="/bin/sed -i 's/^/# /' $1" crontab -e

re-enable crontabs:

VISUAL="/bin/sed -i 's/^# //' $1" crontab -e

46. Checking whether a URL is marked for CDN caching (by going to the origin server):

not cachable: curl -I "https://origin.foobar.com" returns header: Cache-Control: private, max-age=0, must-revalidate

cachable by CDN: curl -I "https://origin.foobar.com" returns header: Cache-Control: max-age=3354, public

47. fuser is akin to lsof

48. dmidecode

e.g., sudo dmidecode -s system-serial-number

49. Use SOCKS proxy to upload over SSH:

curl --upload-file $1 --socks5-hostname CONDUITHOST:1080 http://config/package/

50. single user mode:

select kernel in grub list e edit kernel line, put "single" at end b

51. cat from stdin into a new file:

% cat - > /tmp/foo.txt

control-D (EOF)

52. git: see file at specific revision:

git show e4f434489529854064b5818d4a728806f577326d:./haproxy.cfg.erb | diff - haproxy_rewards.cfg.erb

53. kill -QUIT

Get Java thread stack trace from running process

54. figlet - creating big ASCII text words on the console - ASCII art

figlet -I2 gives directory where fonts are available

e.g., /usr/local/Cellar/figlet/2.2.4/share/figlet/fonts

55. strace on Mac: dtruss -f

NOTE: didn't work well for capistrano commands

56. expanding shell syntax for HOSTS

cap roll HOSTS=echo foo-app{2..5}.colo | tr ' ' ',' MAX_HOSTS=6 ==> use commify:

57. ERB syntax checking

cat haproxy_config/config/templates/haproxy.cfg.erb | erb -x -T - | ruby -c

58. Shell globbing to exclude files:

enable shell opt "extglob" ls hosts/!(pci*)

59. Teeing output to a file:

command 2>&1 | tee logfile

60. Collaborating remotely:

screen skype gotomeeting http://collabedit.com/ https://github.com/Pita/etherpad-lite

61. Viewing only DNS A requests and responses via a Wireshark view filter:

dns.qry.type == A or dns.resp.type

dns.qry.type == A or (dns.resp.type and udp.length == 143)

(check udp.length against the expected length of DNS responses (avoids inclusion of other responses like AAAA))

62. Network just totally broken in Mac, try these things:

• Turned the wireless adapter off and on repeatedly
• Remove /Library/Preferences/SystemConfiguration/com.apple.airport.preferences.plist.
• Create a new Location in Network Preferences.
• If specific networks cannot be joined, find them in Keychain Access, and remove them.

63. mpstat -P ALL 2

64. netstat -su

65. cat /proc/interrupts

66. cat /proc/net/snmp

67. sar: options to enable: -S INT -S DISK -S XDISK -S SNMP (Note that IPv4 statistics depend on sadc option "-S SNMP" to be collected. The following values are displayed (formal SNMP names between square brackets):)

68. Patches w/ git:

git diff --no-prefix > patchfile patch -p0 < patchfile

git diff > patchfile patch -p1 < patchfile

git diff > patchfile git apply patchfile

git format-patch master --stdout > fix_empty_poster.patch

69. git delete tag git tag -d ops-ns_public_config-2012.02.27_15.46 git push origin :refs/tags/ops-ns_public_config-2012.02.27_15.46

70. invoking cmds thru cap

HOSTS=baz COMMAND=hostname cap rolled tat invoke HOSTS=foo,bar COMMAND=whoami cap invoke

71. Loopback mounting an ISO:

mount -t iso9660 source.iso -o loop /mnt/iso_files

72. Equivalent curl & wget cmds for reading a file quietly:

wget -qO- URL curl -s URL

73. Getting "alias" cmds to work over remote SSH

????

74. Mac "Open" dialog shortcuts:

Cmd-/ Allows typing path (e.g., ~/.ssh/config) Cmd-Shift-. Toggles hidden directories

75. Finding and chowning files owned by a specific user

you could use something like "find / -ls | grep someuser" to identify all of that user's files

chown -R someuser: /some/dir

This can be a way to churn through the files if there are a lot or you don't know exactly which files are supposed to be owned by that user:

find -user someuser -exec chown someuser: {} ;

76. Convert unix timestamp into human readable time:

function unixtime() { perl -le "print scalar localtime $1" }

77. Check ruby process size (e.g., w/in test)

    pid, size = ps ax -o pid,rss | grep -E "^[[:space:]]*#{Process::pid}".chomp.split(/\s+/).map {|s| s.strip.to_i} assert(false, "size: #{size}")

78. Install ack (ack-grep) on mac:

http://mdskinner.com/resources/how-install-ack-mac

79. Tool for obtaining info about a server's underlying hardware from SMBIOS:

dmidecode

http://en.wikipedia.org/wiki/Desktop_Management_Interface http://en.wikipedia.org/wiki/System_Management_BIOS

80. Shell locking:

#!/bin/sh LOCKFILE=/var/tmp/s3sync.lock

exec 200>$LOCKFILE if flock --exclusive --nonblock 200; then echo "Got lock. Do something." else echo "Another instance is already running. Exiting." fi

81. getting mtime of a file from unix cmd line:

stat -c %y filename

OR

Linux version of mtime (based on GNU stat)

function mtime() { perl -e "print scalar(localtime(stat -c %Y $1)) . "\n"" }

OR

Mac version of mtime (based on BSD stat)

function mtime() { perl -e "print scalar(localtime(stat -f %m $1)) . "\n"" }

82. Secure erasing data:

ATA has a SECURE ERASE option http://code.google.com/p/diskscrub/ http://www.dban.org/

83. Synchronize 2 local directories using rsync (from 1 to another):

rsync -avz --stats --exclude .git ~/dev/ops-tools-dns-*/ ~/dev/ops-tools-2

84. A10 Networks: disable pager ('more' behavior):

terminal length 0

cut -f1-6,8- filename

86. VIM: change inside containment characters (e.g., #{afdafdasfd})

inside the middle of the containment chars, do: ci{

({ being the containment char)

Similarly, to delete everything inside containment characters, you can switch 'c' with 'd'; e.g., di{

87. Delete photos from iPhone using Mac Application:

Hook up phone to computer using USB cable Use ImageCapture application to select photos to delete Click Red X Icon at bottom left to delete

88. Deploy specific git revision via capistrano:

cap $target deploy -s revision=$sha1

curl ifconfig.me curl ifconfig.me/all/json {"connection":"","ip_addr":"173.241.26.84","lang":"","remote_host":"","user_agent":"curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5","charset":"","port":"59034","via":"","forwarded":"","mime":"/","keep_alive":"","encoding":""}

90. Delete deferred postfix emails:

sudo postsuper -d ALL deferred

(Useful when emails continue to trickle out of a host, e.g., from cronjob problems)

91. Check expiration date of cert

openssl x509 -in cert -noout -enddate

92. ss -l listening sockets (like "netstat -plan | grep -i listen")

sed -i "s/usernameN.local usernameN/$etc_hosts_data/g" /etc/hosts

94. Hand edit /etc/passwd & /etc/group

vipw vigr

96. TODO: string intersection between files via grep (i.e., multiple-grep)

97. Copy all files containing a specific string:

grep -l 'elivered.*foobar' * | xargs -I {} cp {} /tmp/outputdir/.

98. git visual tool: gitk

99. Query DKIM record from DNS:

dig +noall +answer TXT DKIMDNSHOSTNAME.

100. Find WiFi password stored on a Mac:

Go to Applications --> Utilities --> Keychain Access. Search for your wifi network name. Double click the entry that pops up. Click on the show password box.

101. Set operations for string files on unix CLI

http://www.scribd.com/doc/8643762/Set-Operations-in-Unix-Shell e.g., set union: sort set1 set2 | uniq set difference: sort set1 set2 | uniq -u set intersection: sort set1 set2 | uniq -d

102. Get total space used for a set of files:

du -shc file1 file2 file3 | tail -n 1

103. VIM: delete/change from current prompt up to another character

df dt (dt —> delete til) (df —> delete thru ... or "fru"

104. htop

(a) configuration file (~/.htoprc) (b) tips: http://www.thegeekstuff.com/2011/09/linux-htop-examples/ http://www.freesoftwaremagazine.com/articles/htop_tip_top_ncurses_interactive_tool_system_monitoring_your_desktop

105. SQL join explanations:

http://www.codinghorror.com/blog/2007/10/a-visual-explanation-of-sql-joins.html

106. Safari tab selection (making it suck less):

https://github.com/rs/SafariTabSwitching

107. Better Touch Tool for Mac to allow snapping and auto-resizing of windows:

http://blog.boastr.net/?page_id=1722

108. UTC Menu Clock:

https://github.com/netik/UTCMenuClock NOTE: I have a modified version from Andrew Ho that has no ticking-seconds: it's in my dropbox account, in the "Mac Programs" folder

109. Enable Safari Debug menu:

(Terminal)# defaults write com.apple.Safari IncludeInternalDebugMenu 1 Then relaunch safari

110. Function in Google Docs Spreadsheet (using IF with AND):

=IF(AND(B2 = "snake", C2 <> "yes"), "YES", "no")

111. Search in Mac Finder: default to "this folder", instead of "this mac":

http://superuser.com/questions/365731/finder-how-to-make-default-search-in-folder-the-current-folder

112. Do not put % into cron lines; e.g., $(($RANDOM%30))

113. Handle gigantic dirs (tons of contained files) http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/

114. Typing various chars on Mac:

http://tlt.its.psu.edu/suggestions/international/accents/codemac.html

¿: shift-option-? ñ: alt-n,n

115. yawk trick: avoid lots of if logic with rescue

Perhaps this is super obvious to everybody but me, but this empty rescue statement saved me a ton of "if" logic, which is ugly when concatenated together in a one-liner:

bin/yawk 'begin puts "#{d.hostname}.#{d.colo}: " + d.monitors.iostat.shell_command; rescue; end' hosts/*

Or: "Show me the hostname.colo and shell command of all hosts which have overridden the iostat monitor from their hostclass" This can be shortened even further by using the "inline rescue"...

bin/yawk 'puts "#{d.hostname}.#{d.colo}: " + d.monitors.iostat.shell_command rescue nil' hosts/*

116. To edit a cmdline string directly in vim

ctrl-x ctrl-e EDITOR=vim

117. Apache: disable keepalives (to avoid requiring clients to send "Connection: close" HTTP header)

http://httpd.apache.org/docs/2.2/mod/core.html#keepalive

118. Looking for logs in Mac? Console.app is your friend.

119. Ruby Gem File inclusion failure:

require 'net/ssh' not working for ya? How about require 'whatever'?

Could be that you need to require 'rubygems' first!

http://docs.rubygems.org/read/chapter/3
http://guides.rubygems.org/what-is-a-gem/

120. generating email and sending over SMTP swaks: swiss army knife smtp e.g., ~/bin/swaks --to someone@example.com --server SMTPHOSTNAME --port 2400 smtp-source: send crafted smtp messages (part of postfix) xtress: load testing suite for smtp

121. Get vi-mode in bash to work better:

http://www.jukie.net/bart/blog/20040326082602

^l clear screen

bind -m vi-insert "\C-l":clear-screen

allow all readline clients to use vi-mode too

set editing-mode vi set keymap vi set convert-meta on

122. Gmail slowness?

• disable/remove boomerang browser add-on
• disable the "3rd party access" authorization within your account for b4g.baydin.com (that's boomerang for gmail's domain): http://support.google.com/accounts/bin/answer.py?hl=en&answer=41236
• disable ALL mail applications that were using IMAP to access gmail. For me that meant Mail.app on MacBook, iPad, iPhone. For iPad, iPhone, just did this: Settings > Mail, Contacts, Calendars > myaccountname > Mail > Off

123. pdsh : distributed ssh, executes cmd on multiple hosts

pdsh -w foohost[1-6].colo df -h

124. Finding memory hog proceses:

watch -n 30 top -c -o res -b >> mem.log

125. Clear / delete / remove IP address completely from interface

So, if you have a pre-existing IP config on an interface, say eth1, and then you just remove that config, the IP config will linger on (visible in output of both ifconfig && ip addr), even though there is no ifcfg-eth1 script in /etc/sysconfig/network-scripts To fully remove the IP address from the interface, it is insufficient to just restart networking, nor can you just down and up the interface. Nor can "ifconfig del" help you. You need to use "ip addr del", or you need to set the IP to 0.0.0.0: http://superuser.com/questions/153559/clear-ip-address-of-ethernet-interface i.e., ip addr del 1.2.3.4 dev eth1 ifconfig eth1 0.0.0.0

126. du and df being out of sync: Explanation:

du will not show space used by files that are deleted from the filesystem (e.g., with rm)
but are still being held by a process, since they are not reachable thru the directory
tree anymore.  But the space is not reclaimed by the filesystem until all references to a
file are gone.

df will still show the space as being used until the references are gone.

Find the dangling references:

sudo lsof | grep '(deleted)'

127. Getting cmd line text into Mac's paste buffer:

echo "foo" | pbcopy

128. Make Safari less sucky:

http://www.machangout.com

129. getting info on what the CPU/IO/memory is doing:

mpstat -P ALL 1 iostat -mtx 1 vmstat 1 htop

130. split CSV list into separate lines with line-number prefixes:

     % echo 'foo,bar,baz' | tr ',' '\n' | nl -ba -s') ' 1) foo 2) bar 3) baz

131. working with git branches:

http://stackoverflow.com/questions/1519006/git-how-to-create-remote-branch http://sandofsky.com/blog/git-workflow.html

132. paste cmd to combine multiple lines

echo "foo\nbar" | paste -s -d' ' -

133. use awk as the opposite of paste cmd; i.e., to rip a each line from a file apart into multiple lines

echo 'email-msys1 10.0.0.1' | awk '{print "- address: " $2":2525", "\n name: " $1}'

134. storing text records in DNS that are greater than 512 bytes

20120113._domainkey.google.com. 86400 IN TXT "k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAp5kQ31/aZDreQqR9/ikNe00ywRvZBFHod6dja+Xdui4C1y8SVrkUMQQLOO49UA+ROm4evxAru5nGPbSl7WJzyGLl0z8Lt+qjGSa3+qxf4ZhDQ2chLS+2g0Nnzi6coUpF8r" "juvuWHWXnzpvLxE5TQdfgp8yziNWUqCXG/LBbgeGqCIpaQjlaA6GtPbJbh0jl1NcQLqrOmc2Kj2urNJAW+UPehVGzHal3bCtnNz55sajugRps1rO8lYdPamQjLEJhwaEg6/E50m58BVVdK3KHvQzrQBwfvm99mHLALJqkFHnhyKARLQf8tQMy8wVtIwY2vOUwwJxt3e0KcIX6NtnjSSwIDAQAB"

135. Find which filesystem a file or directory resides within:

[root@hostname log]# df -PTh /some/dir/log Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 48G 42G 3.0G 94% /

[root@hostname /]# df /some/dir/log Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb1 1133279828 536728136 538055816 50% /data

136. To get all git tags

git fetch --tags

git pull does not get all tags but only those reachable from the current branch heads. However, git pull --tags fetches all tags and is apparently equivalent to git fetch --tags.

137. Calculate time difference with respect to timezones

http://www.wolframalpha.com/input/?i=13+hours+54+minutes+before+4%3A22AM

138. Run repeated DNS queries 10 times a second, saving the timing and result to a file & stdout using tee:

while true; do date && (time dig @10.20.253.33 A DNSHOSTNAME +short +timeout=30) 2>&1 | tee -a dns.log; sleep 0.1; done

139. Use dig to perform reverse DNS lookup of 50.115.214.1:

% dig +short PTR 1.214.115.50.in-addr.arpa mta1l1.r.grouponmail.co.uk.

140. Strace

sudo strace -tt -s 1024 -o strace.log -ffv -x -p 16802

-x ==> print all non-ASCII in hex format

141. File where network preferences are stored on Mac (as of Lion):

/Library/Preferences/SystemConfiguration/com.apple.airport.preferences.plist

140. Join all lines in a file

cat file | tr -d '\n'

141. DKIM validator tool

https://9vx.org/~dho/dkim_validate.php

142. Convert unix timestamp to human-readable time:

date -d @1360370800

curl --socks5-hostname conduit:1084 http://grapher/dashboard/144

144. Does not exist: DNE: ¬∃

145. Named / bind debugging:

sudo rndc dumpdb -all

146. putting process in background:

control-z bg disown -h

147. find files of at least X bytes or at most X bytes (example use 4KB): at least: find . -type f -size +4096c at most: find . -type f -size -4096c

148. list all files with contents from a dir

find . -type f -print | while read filename; do echo -n "$filename: " ; cat "$filename" ; done

149. how to daemonize a process:

http://www.itp.uzh.ch/~dpotter/howto/daemonize

To properly daemonize, the following steps must be followed.

The fork() call is used to create a separate process.
The setsid() call is used to detach the process from the parent (normally a shell).
The file mask should be reset.
The current directory should be changed to something benign.
The standard files (stdin,stdout and stderr) need to be reopened.

Failure to do any of these steps will lead to a daemon process that can misbehave. The typical symptoms are as follows.

Starting the daemon and then logging out will cause the terminal to hang. This is particularly nasty with ssh.
The directory from which the daemon was launched remains locked.
Spurious output appears in the shell from which the daemon was started.

150. Frank's compulsory guide to postal addresses:

http://www.columbia.edu/~fdc/postal/

Lists address formats for most (all?) countries

151. calling shell command (e.g., foo) directly without aliases or functions:

command foo

152. problems with safari & fuzebox

Finder > Go > Hold Option & Click Library Search for "safari" remove all of the plist files

153. pkill without pkill: # From: http://en.wikipedia.org/wiki/Kill_(command)

kill ps --no-headers -C firefox -o pid

154. stop using "bundle exec"

https://github.com/mpapis/rubygems-bundler

155. too many sockets in CLOSE_WAIT state (e.g., in unicorn)

$ sudo lsof | grep CLOSE_WAIT | wc -l 27405

What is the other end of the CLOSE_WAIT sockets? Could you paste a sampling of "sudo netstat -alnp | grep CLOSE_WAIT | grep unicorn"? Does the situation change if you disable keepalive at the nginx or app level? (send "Connection: close" header)

156. What process string do pkill & pgrep & killall search through?

http://askubuntu.com/questions/27501/whats-the-difference-between-killall-and-pkill

By default:

/proc/<PID>/stat

If you use "-f" with pkill & pgrep, then it looks through: If you use "

/proc/cmdline

157. Comparing version numbers in ruby

http://stackoverflow.com/questions/2051229/how-to-compare-versions-in-ruby

Gem::Version.new('0.4.1') > Gem::Version.new('0.10.1')

richer: https://github.com/dazuma/versionomy

158. Use tcp device to make HTTP requests in the shell, just using exec, echo, & cat

exec 3<>/dev/tcp/www.groupon.com/80 && echo -e "GET / HTTP/1.1\r\nhost: http://www.groupon.com\r\nConnection: close\r\n\r\n" >&3 && cat <&3

159. Use iftop with tcpdump filters to find if traffic has stopped to a service

sudo iftop -P -N -f 'dst portrange 5000-5049'

160. mou --> good for markdown wysiwyg editing, and then you can use pandoc to convert to mediawiki

161. Mac Top Menu Bar frozen?

killall SystemUIServer

http://apple.stackexchange.com/questions/39219/what-should-i-do-when-the-menu-bar-seems-frozen-unresponsive

162. Tail a file that hasn't yet been created:

tail --follow=name

163. Determining which processes have various TCP connections on Mac (since "netstat -pan" fails)

lsof -n -i4TCP -P

164. memcache top

mctop

165. prevent logins

sudo touch /etc/nologin (Actually, need to double check that, I think it varies greatly between BSD and Linux)

166. Shell enhancements for Git (zsh, bash):

http://nuclearsquid.com/writings/git-tricks-tips-workflows/ http://stackoverflow.com/questions/1128496/to-get-a-prompt-which-indicates-git-branch-in-zsh

http://dmytro.github.io/2012/11/27/unobtrusive-git-prompt.html

http://stackoverflow.com/questions/6623649/disable-auto-completion-of-remote-branches-in-git-bash/

http://stackoverflow.com/questions/12175277/disable-auto-completion-of-remote-branches-in-zsh

167. Make yourself admin on a Mac (Mountain Lion)

I just got a new MBAir from IT and they didn't set my account up as a computer admin. This made it so I couldn't change security settings (like FileVault), set up the VPN, or install certain software.

The best solution is to have IT set you up as an admin, but if that's not an option (I'm working from home today), here's how to do it:

1. shut down the computer
2. boot into single-user mode by holding down CMD-s as you power on the computer
3. when it's done booting, run these commands:

fsck -fy

mount -uw /

launchctl load /System/Library/LaunchDaemons/com.apple.opendirectoryd.plist

sudo dseditgroup -o edit -a yourlogin -t user admin

sudo dseditgroup -o edit -a yourlogin -t user wheel

reboot

4. When the machine is done booting, log in and now you're an admin!

http://steinkamp.us/934/single-user-admin-on-os-x-mountain-lion

168. Get your host's public-visible NAT IP

curl icanhazip.com

wget -q -O- icanhazip.com

169. Avoid clearing/polluting disk cache with a periodic process that scans a filesystem

nocache ionice -c3 nice long_running.sh

http://packages.debian.org/sid/nocache

170. Avoid data changes being synced to disk

eatmydata

http://packages.debian.org/sid/eatmydata

171. Set default shell in mac:

chsh -s /bin/zsh chsh -s /usr/local/bin/fish # http://fishshell.com/

172. Combine output from 2+ cmds into 1 stream by using a sub-shell which concatenates output into 1 stream:

(foo; bar;) | baz

Tests of it not being interleaved:

(yes foo | head -n 50000; yes bar | head -n 50000;) > /tmp/5 (seq 0 99999; seq 1000000 1099999) > /tmp/5

173. The "suck" effect for minimizing a window on OS X You can choose between the "scale" or "genie" effect when you minimize a window on your Mac. But there's a secret third choice called "suck." Start up Terminal and type "defaults write com.apple.dock mineffect -string suck" and restart your computer. The new effect is activated.

Read more: http://www.businessinsider.com/mac-os-x-easter-eggs-2013-4?op=1#ixzz2n14Nkjf6

174. copy and paste issues on Mac By default: command-shift-v : Paste and Match Style

Can supposedly change to command-v doing "Paste and Match Style" by doing this:

defaults write .GlobalPreferences -dict-add NSUserKeyEquivalents "Paste and Match Style" -string "@v"

http://www.macstories.net/tutorials/customize-your-macs-paste-and-match-style/

175. Emoji unicode characters that render as crossed out mushroom:

http://graphemica.com/search?q=%F0%9F%8D%84%E2%83%A0

176. Get first and last lines from the output of a command

(curl -s -i http://localhost:8999/heartbeat.txt | tee /dev/fd/3 | head -n 1) 3> >(tail -n 1)

http://stackoverflow.com/questions/8624669/unix-head-and-tail-of-file

177. Sound an alert from a terminal (causes beep sound and puts a bell icon in Terminal.app tab title)

printf '\7'

178. Smart Questions: follow-up with solution:

http://www.catb.org/~esr/faqs/smart-questions.html#followup

179. Install specific version of something in brew (e.g., to downgrade git):

http://stackoverflow.com/questions/3987683/homebrew-install-specific-version-of-formula

180. cmds that go through pager have escape codes instead of colors

http://unix.stackexchange.com/questions/64927/git-diff-displays-colors-incorrectly

git config --global core.pager 'less -R'

181. slabtop

```
slabtop
```

Figuring out what "kernel slab cache" memory is being used for.
http://www.toofishes.net/blog/linux-command-day-slabtop/

182. Get a file's rename history in git

```
git log --format='%H%%' --name-only --follow -- somefile/in/your/git/repo | gsed ':a;N;$!ba;s/%\n\n/ /g' | awk '{print $2}' | uniq
```

183. Work with JSON on the command line.

```
jq
# e.g., sort the keys
jq -S '.'
```
http://stedolan.github.io/jq/tutorial/

184. Run previous command with sudo

```
sudo !!
```

185. Substitute in previous command's final argument

```
!$
```

186. Get the number of connections to a port; e.g., 2181

```
% ss -a -t -n state connected src :2181 | wc -l
37
```

187. Bash shell expansion

http://wiki.bash-hackers.org/syntax/expansion/brace
```
foo{1..15}
foo[1-9]  # cannot do foo[1-15], since that expands to foo1,foo5; i.e., the square bracket syntax is useful only with single characters
```

188. echo -n is bad emkay, use printf "%s" instead

https://github.com/git/git/commit/19c3c5fdcb35b66b792534c5dc4e8d87a3952d2a

POSIX-compatible, portable method for printing a string without a newline is:
```
printf "%s" "text"
```
instead of
```
echo -n "text"
```

189. github / git tips and tricks

http://owenou.com/2012/01/13/ten-things-you-didnt-know-git-and-github-could-do.html

190. Show definition of a shell alias or function:

zsh:
```
type some-alias
whence -f some-alias
```

bash:
```
type some-alias
```

191. Curl cmd that provides detailed timing:

when you test it again, perhaps try this curl command which provide more timing details:

```
curl -o /dev/null -s -w
"time_namelookup: %{time_namelookup}\ntime_connect:
%{time_connect}\ntime_appconnect:
%{time_appconnect}\ntime_pretransfer:
%{time_pretransfer}\ntime_redirect:
%{time_redirect}\ntime_starttransfer:
%{time_starttransfer}\ntime_total: %{time_total}\n"
http://somehost/
```

192. Git: reverting a commit

http://stackoverflow.com/questions/4114095/revert-to-a-previous-git-commit

193. Get age of a process

```
stat /proc/$pid
```

194. Weird behavior of exempting "grep" from process searching:

``` 
% ps aux | grep [/]bin/bash | grep grep
% ps aux | grep /bin/bas[h] | grep grep
1226     22256  0.0  0.0  61192   828 pts/20   S+   01:59   0:00 grep /bin/bash
``` 

  The [] is treated as a shell glob initially, with the shell trying to
  expand the argument to the full matching file name (which /bin/bash *is*).

http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_04.html#sect_03_04_08

195. Make colluquy shut up about joined/left messages

http://colloquy.info/project/wiki/FAQs#Doescolloquyhaveawaytoignorehidepartsnicksetcinabusyroom
http://wiki.xkcd.com/irc/Hide_join_part_messages#Colloquy

196. Controlling tabs in Mac Terminal

http://superuser.com/a/176181/261282

http://www.culater.net/software/SIMBL/SIMBL.php

197. Figure out which java thread (in jstack) a Linux Thread corresponds to

http://www.semicomplete.com/blog/geekery/debugging-java-performance.html

jstack PID > stacktrace

# have PID for Linux thread you care about (e.g., from running strace), convert that to hex:

% printf "0x%x\n" 28993
0x7141

% grep nid=0x7141 stacktrace
"VM Thread" prio=10 tid=0x000000005e6d9000 nid=0x7141 runnable 

198. Find which package a file in centos/redhat came from:

rpm -qf <filename>

199. Create file with random bits of size X

dd if=/dev/urandom of=2kB.rand bs=1K count=2

200. Latency numbers of various I/O operations

https://gist.github.com/jboner/2841832

Latency Comparison Numbers
--------------------------
L1 cache reference                            0.5 ns
Branch mispredict                             5   ns
L2 cache reference                            7   ns             14x L1 cache
Mutex lock/unlock                            25   ns
Main memory reference                       100   ns             20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy              3,000   ns
Send 1K bytes over 1 Gbps network        10,000   ns    0.01 ms
Read 4K randomly from SSD*              150,000   ns    0.15 ms
Read 1 MB sequentially from memory      250,000   ns    0.25 ms
Round trip within same datacenter       500,000   ns    0.5  ms
Read 1 MB sequentially from SSD*      1,000,000   ns    1    ms  4X memory
Disk seek                            10,000,000   ns   10    ms  20x datacenter roundtrip
Read 1 MB sequentially from disk     20,000,000   ns   20    ms  80x memory, 20X SSD
Send packet CA->Netherlands->CA     150,000,000   ns  150    ms
 
Notes
-----
1 ns = 10-9 seconds
1 ms = 10-3 seconds
* Assuming ~1GB/sec SSD
 
Credit
------
By Jeff Dean:               http://research.google.com/people/jeff/
Originally by Peter Norvig: http://norvig.com/21-days.html#answers
 
Contributions
-------------
Some updates from:                      https://gist.github.com/2843375
Great 'humanized' comparison version:   https://gist.github.com/2843375
Visual comparison chart:                http://i.imgur.com/k0t1e.png
Nice animated presentation of the data: http://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development/

201. Test for whether connections are open

% for conn in {"foo 22","foo 80","foo 443","bar 9418"}; do nc -z -w1 $conn; if [ "$?" -ne 0 ]; then echo "$conn: FAILED, port not open"; fi; done
Connection to foo 22 port [tcp/ssh] succeeded!
foo 80: FAILED, port not open
Connection to foo 443 port [tcp/https] succeeded!
Connection to bar 9418 port [tcp/git] succeeded!

202. Running diff of lsof each time you run it

0 - chi@kafka-hadoop-consumer2:$ for i in {1..10}; do sudo lsof -p 3800 > /tmp/lsof$((i % 2)) && diff /tmp/lsof0 /tmp/lsof1 >> /tmp/lsof-delta.txt; done diff: /tmp/lsof0: No such file or directory 1 - chi@kafka-hadoop-consumer2:$ tail /tmp/lsof-delta.txt

java 3800 root *531r FIFO 0,6 0t0 97169587 pipe java 3800 root *532w FIFO 0,6 0t0 97169587 pipe java 3800 root *533r 0000 0,11 0 97169588 eventpoll 41542a41552,41553 java 3800 root *537w FIFO 0,6 0t0 97172727 pipe java 3800 root *538r 0000 0,11 0 97172728 eventpoll 41544a41556,41558 java 3800 root *546r FIFO 0,6 0t0 97170919 pipe java 3800 root *547r FIFO 0,6 0t0 97170919 pipe java 3800 root *548w 0000 0,11 0 97170920 eventpoll

203. Stand up a simple HTTP server for serving files:

```
python -m SimpleHTTPServer 4004

# OR

ruby -run -e httpd . -p 5000

```

http://www.linuxjournal.com/content/tech-tip-really-simple-http-server-python
http://www.benjaminoakes.com/2013/09/13/ruby-simple-http-server-minimalist-rake/

204. Trim in shell using sed

sed 's/^ *//;s/ *$//'

205. OS X tweaks

https://www.learningosx.com/101-ways-to-tweak-os-x-using-terminal/

206. Cannot put multiple ~ chars inside a single string in a shell and except them to be expanded; e.g., in a Java classpath with multiple jars.

http://stackoverflow.com/questions/9122989/use-javac-with-multiple-specific-jars-in-linux

https://bugs.openjdk.java.net/browse/JDK-4668475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel

207. Debugging class loading issues

java -verbose:class

http://www.ibm.com/developerworks/library/j-dclp1/

208. Find traffic between 1 host and not some others:

sudo iftop -i eth2 -f 'port 2181 and not host foobar and not host baz'

209. Reading through files with gigantic lines (with many many more characters than fit in your terminal's width)

| less -S | cut -c1-200

Or whatever your terminal's width in characters is

210. Pastebin CLI cmds for Mac

pbcopy pbpaste

211. Take CSV and make it readable

cat foo | column -s, -t column -s, -t < foo

Alternatives:

1. $ csvgrep -e iso-8859-1 -c 1 -m "de" worldcitiespop | csvgrep -c 5 -r "\d+" | csvsort -r -c 5 -l | csvcut -c 1,2,4,6 | head -n 11 | csvlook

i.e., from csvkit: https://csvkit.readthedocs.org/en/latest/

2. csvtool readable filename | view -

3. csv plugin for VIM: http://vim.sourceforge.net/scripts/script.php?script_id=2830

4. Tabview : https://github.com/firecat53/tabview

5. bash with various file descriptors using exec

http://www.tldp.org/LDP/abs/html/x17974.html

See LOCKFILE stuff above.

213. Switch java version

export JAVA_HOME=$(/usr/libexec/java_home -v1.8)

215. Fancy use of /proc

https://blogs.oracle.com/ksplice/entry/solving_problems_with_proc

216. Lots of good bash stuff:

http://stackoverflow.com/questions/2188199/how-to-use-double-or-single-bracket-parentheses-curly-braces

217. Building a JSON-based API:

http://jsonapi.org/

including great stuff for pagination: http://jsonapi.org/format/#fetching-pagination

218. All about redirects, including exec with redirects

http://wiki.bash-hackers.org/howto/redirection_tutorial https://stackoverflow.com/questions/18351198/what-are-the-uses-of-the-exec-command-in-shell-scripts

219. Prefer long-form bash redirects

i.e., Never use the Csh &>foo shorthand redirect. Use the long form >foo 2>&1.

Use the long form 1>foo 2>&1.

221. Get a process's env:

% sudo sh -c 'xargs --null --max-args=1 < /proc/26672/environ' LD_LIBRARY_PATH=/usr/local/lib:/usr/local/jre/lib/amd64/server MESOS_LOG_DIR=/var/logs/mesos MESOS_CLUSTER=some_cluster_name MESOS_IP=192.168.0.1 MESOS_QUORUM=2 MESOS_WORK_DIR=/var/blah/mesos/work_dir MESOS_ZK=zk://zookeeperhostname:2181/mesos_dev

222. Undo last git commit:

git reset --soft HEAD^

223. Ansible use of paramiko vs. native OpenSSH

When switching between source hosts for running ansible you might notice differing behaviors w.r.t. ansible respecting your ~/.ssh/config options. This happens because ansible (after 1.2) tries to use OpenSSH if it's version 5.6+, and if the version of OpenSSH is too old then ansible falls back to using paramiko, which doesn't respect the ~/.ssh/config parameters like OpenSSH does. Notably, Centos6 doesn't have OpenSSH 5.6+.

To overcome this, add the following section to ~/.ansible.cfg:

[defaults]
host_key_checking = False

http://docs.ansible.com/ansible/intro_getting_started.html#remote-connection-information
http://docs.ansible.com/ansible/intro_getting_started.html#host-key-checking

224. ZooKeeper: delete nodes recursively in zookeeper-3.3*

https://github.com/davidledwards/zookeeper/tree/master/zookeeper-cli

225. Capture packets remotely using tcpdump & wireshark:

wireshark -k -i <(ssh REMOTE_HOST "sudo tcpdump -w - port 6557")

226. Use sed to clean up known_hosts

gsed -i '/staas-worker.*staging/d' ~/.ssh/known_hosts

227. get CPU usage info for a PID:

ps -p PID -o cputime ; ps -p PID -o etime ; ps -p PID -o %cpu

NOTE: cputime is cumulative across all cores, so can be more than elapsed time (clock time)

228. List threads owned by monitord

ps u -L -U monitord

229. Get actual cpu usage; ps's %cpu is cummulative for time process has run:

ps -o%cpu,cputime,etime -p 28038

i.e., %cpu by default is just etime / cputime
so really, we need to take the difference between 2 snapshots of cputime over a 
period of time to get the usage during that period

TODO: write a wrapper for doing that

230. Vim: substitute newline

use \r instead of \n

https://stackoverflow.com/questions/71323/how-to-replace-a-character-for-a-newline-in-vim https://stackoverflow.com/questions/71417/why-is-r-a-newline-for-vim

231. tar pipe: copy a bunch of files quickly over SSH

PUSH: from this host to another one

tar czf - kernel | ssh REMOTEHOST "cd tmp/ ; tar xzf -"

PULL: from remote host to this host:

ssh REMOTEHOST "tar czf - kernel" | tar xzf -

PULL: with root-owned files

ssh REMOTEHOST "cd /etc ; sudo tar czpf - ssh" | sudo tar xzpf -

where kernel & ssh are dirs with a bunch of files

232. Mirror Displays on Mac via keyboard shortcut: Command + F1

233. Google+ comment cheatsheet:

https://docs.google.com/document/d/1tkKOd0wKsQLKn5Dtkqy8p_DxQ0ikZTXJohzIfw-_gNQ/edit

234. Bash loops to search for processes with certain environment variables

while true; do sudo sh -c 'strings /proc//environ | grep MESOS | grep -v SUDO_COMMAND'; done while true ; do for p in /proc/[0-9]; do sudo sh -c "strings $p/environ | grep MESOS && cat $p/cmdline ; strings $p/environ | grep MESOS" ; done; done;

235. Bash check exit values

http://blog.sanctum.geek.nz/testing-exit-values-bash/

Bad practice

grep -q regex options if (($? > 0)); then printf '%s\n' 'myscript: Pattern not found!' >&2 exit 1 fi

Better

if grep -q regex options; then # Do nothing : else printf '%s\n' 'myscript: Pattern not found!\n' >&2 exit 1 fi

Best

if ! grep -q regex options; then printf '%s\n' 'myscript: Pattern not found!' >&2 exit 1 fi

236. Get Instruction Pointer (32-bit: EIP, 64-bit: RIP) for Linux threads:

ps -eo tid,pid,rip,state -L

237. Container visibility: sysdig / csysdig

https://github.com/draios/sysdig/wiki/Csysdig-Overview http://www.sysdig.org/install/

sudo yum -y update kernel curl -s https://s3.amazonaws.com/download.draios.com/stable/install-sysdig | sudo bash

238. Enter container with same PID & mount namespaces as specified PID:

sudo nsenter -p -m -t

239. Reset local git branch to be same as remote branch (useful after someone force-pushes):

git reset --hard /

e.g., git reset --hard DarinJ/0.10.0

https://stackoverflow.com/questions/1628088/reset-local-repository-branch-to-be-just-like-remote-repository-head

240. Find processes running at a given instant

grep -E '^R.*' /proc/sched_debug

241. Git tips / scripts

http://nuclearsquid.com/writings/git-tricks-tips-workflows/ https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/ https://wiki.typo3.org/Tips_and_Tricks_%28Git%29

242. HEREDOC for pasting file to be processed

% column -t << EOL heredoc> iface bytes packets heredoc> lo 86654545 615357
heredoc> eth0 6096258286 40319326 heredoc> eth1 6119552036 40693237 heredoc> bond0 12215810436 81012564 heredoc> br0 2222175609 11065906 heredoc> vnet0 15989996953 73543679 heredoc> EOL iface bytes packets lo 86654545 615357 eth0 6096258286 40319326 eth1 6119552036 40693237 bond0 12215810436 81012564 br0 2222175609 11065906 vnet0 15989996953 73543679

https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash

243. Use ansible to perform scp with root-privileges

Copy to remote host using sudo ansible -i HOST, -s -m copy -a "src=SRC dest=DEST" HOST

Copy from remote host using sudo ansible -i HOST, -s -m fetch -a "src=SRC dest=DEST flat=yes" HOST

243. Using ansible on localhost

http://ansible.pickle.io/post/86598332429/running-ansible-playbook-in-localhost

244. Use maven to run just the tests, and bail after the 1st error:

mvn surefire:test -DskipAfterFailureCount=1

need surefire 2.19+

245. When using maven clojure plugin, the test failures are squelched unless you run:

mvn clojure:test

i.e.,  `mvn test` is insufficient.

http://alexott.net/en/clojure/ClojureMaven.html

http://www.chaomancy.com/blog/clojure-in-eclipse-part-1-maven.html

246. Rename branch in git:

https://gist.github.com/lttlrck/9628955

git checkout old_branch                     # Ensure you are on the to-be-renamed branch
git branch -m new_branch                    # Rename branch locally
git push origin :old_branch                 # Delete the old branch
git push --set-upstream origin new_branch   # Push the new branch, set local branch to track the new remote

245. Overriding SSH config per-host for a different SSH private key:

Host 54.177.87.70 50.18.145.148 184.72.37.34 geespot-dev geespot-prod geespot-dev.willinglydumb.com geespot-prod.willinglydumb.com IdentityFile ~/.ssh/geespot/id_rsa

246. Clean up logs older than N minutes:

find /some/path/ -type f -mmin +240 -name '*.log' -delete

247. Clean up all dirs with specific path prefix:

find /tmp -path '/tmp/render-sandbox.*' -delete"

248. tig: git curses thing kinda like gitk

249. git : clean up repo

git gc --prune=now git reflog expire --all --stale-fix git prune

250. tcpflow: create separate connections out of pcap

251. pretty-print json

jq . ~/.vagrant.d/data/machine-index/index

252. GitHub permalinks to files:

https://help.github.com/articles/getting-permanent-links-to-files/

type "y"

253. Understanding signals, jobs, process groups; why Ctrl-C might be different than kill -2

https://unix.stackexchange.com/questions/45426/why-would-ctrl-c-behave-differently-than-kill-2

254. Track all processes

psacct acct https://www.gnu.org/software/acct/ http://www.tecmint.com/how-to-monitor-user-activity-with-psacct-or-acct-tools/ http://www.tldp.org/HOWTO/text/Process-Accounting https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#safe=off&q=linux+acct http://www.dedoimedo.com/computers/linux-cool-hacks.html

255. Filter logs based on timestamp

https://stackoverflow.com/questions/33000824/filter-linux-logs-based-on-unix-timestamp

256. Human readable timestamp from logs with unix timestamps

https://www.commandlinefu.com/commands/view/8784/read-squid-logs-with-human-readable-timestamp

257. Colorize terminal environment

https://unix.stackexchange.com/questions/148/colorizing-your-terminal-and-shell-environment

IntelliJ IDEA > Preferences > Editor > General > "Change font size (Zoom) with Ctrl+MouseWheel"

259. ls with full timestamp including milliseconds

% ls -altr --time-style=full-iso

260. Edit paste-buffer contents before pasting into iTerm

Command-Option-v

261. jq usage with mesos:

curl -o- 'MESOSWORKERHOST:5051/slave(1)/state.json' | jq -C ".frameworks[]" | less -R

262. 4-finger swipe / gesture for trackpad stops working.

killall Dock

263. Splunk - less-ish behaviors:

264. Get nearby events: click on event time - "Nearby Events" + "Events Before or After"

265. Get events in reverse (original) order: "| reverse"

266. Bash: [] vs.

https://serverfault.com/questions/52034/what-is-the-difference-between-double-and-single-square-brackets-in-bash

265. Buffer output per-line

stdbuf -oL -eL

266. Firefox telemetry info, including histograms

about:telemetry

267. Same source port held by multiple processes? Probably inherited.

268. Use pmap when trying to look into JVM memory usage

269. Get process start time

ps -eo pid,lstart,cmd | grep

270. watch of a pipe of commands: 'foo | bar'. Also "ps" output gets cut off, pass "www"

sudo watch -d 'ps auxwww | grep logstas[h]'

271. Find and delete files that are not held open:

find * ! -exec sudo fuser -s "{}" 2>/dev/null ; -exec echo {} ; | xargs -I{} sudo rm {}

272. Find and delete files that have not been modified within last hour:

find * -name "strace.log.*" -type f -mmin +60 -delete

273. Mac video camera stuck on laptop cannot use external display

sudo killall VDCAssistant

274. Find size of files excluding some dir

sudo find /some/dir -type f -not -path "/some/dir/baz/*" -exec echo {} ; | sort | xargs -I{} sudo sh -c 'du -b {}' > ~/some-dir-except-baz.du

275. Extract file to stdout from jar

unzip -q -c stormjar.jar META-INF/maven/org.apache.storm/storm-kafka/pom.xml | grep <version | head -n1

276. paste and bc for adding numbers that are in separate lines in a file / stream

cat numberfile | paste -sd+ - | bc

URL="http://www.google.com" ; while true ; do sleep 5 ; echo -n "$(date) " ; if curl -sSf ${URL} >& /dev/null; then echo "${URL} OK" ; else echo "${URL} FAILED" ; fi ; done

278. to ensure proper script behavior follow these tips:

For script authors: Every bash script that uses the cd command with a relative path needs to call unset CDPATH, or else it may not work correctly. Scripts that don’t use cd should probably do it anyway, in case someone puts a cd in later.

For users: Never export CDPATH from your shell to the environment. If you use CDPATH then set it in your .bashrc file and don’t export it, so that it’s only set in interactive shells.

https://issues.apache.org/jira/browse/STORM-2486

https://bosker.wordpress.com/2012/02/12/bash-scripters-beware-of-the-cdpath/

STORM-2486: Prevent cd from printing the target directory.

unset CDPATH

279. Create custom homebrew install

https://github.com/Homebrew/legacy-homebrew/issues/40987

280. git diff

% git diff -W 0.9.0.1:storm-core/src/clj/backtype/storm/daemon/common.clj v1.0.6:storm-core/src/clj/org/apache/storm/daemon/common.clj

281. Enable Picture-in-Picture mode in Mac Sierra for YouTube videos: Step 1: Right click on the YouTube video that you wish to detach Step 2: Once the menu appears, right-click once more to reveal a second menu and then click Enter Picture-in-Picture For normal stuff: https://support.apple.com/en-us/HT206997

282. Show hidden files toggle for Finder

COMMAND + SHIFT + .

283. Stupid xfinity autologin crap causing my devices to think my xfinity service is the current house's xfinity service instead of my own. Disable Auto Authentication https://idm.xfinity.com/myaccount/userprofile

Auto Authentication

  Automatically access TV content while connected to Xfinity Internet at home without having to sign in. Visit the FAQ section to learn more.

284. diff-so-fancy

https://github.com/so-fancy/diff-so-fancy#screenshot

285. IntelliJ help with shortcuts & commands

Command-Shift-A

286. Use Preview.app on Mac to join PDF files together

https://support.apple.com/en-us/HT202945

287. GitHub Search options for not searching only in the base fork & default (usually master) branch

fork:true fork:only