From 32e5f9e24c1e098c4ebaadf5c608a22ff14f8b0e Mon Sep 17 00:00:00 2001 From: Charles Simchick Date: Mon, 8 Jun 2020 17:07:27 -0700 Subject: [PATCH 1/3] Add info on 'rclone' for remote cloud storage file management. --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index ff73938..153446c 100755 --- a/README.md +++ b/README.md @@ -268,6 +268,8 @@ EOF mkdir empty && rsync -r --delete empty/ some-dir && rmdir some-dir ``` +- You can use [`rclone`](https://rclone.org/downloads/) to manage files on remote cloud storage services, including backup/restore, encrypt/decrypt, sync. + - For monitoring progress when processing files, use [`pv`](http://www.ivarch.com/programs/pv.shtml), [`pycp`](https://github.com/dmerejkowsky/pycp), [`pmonitor`](https://github.com/dspinellis/pmonitor), [`progress`](https://github.com/Xfennec/progress), `rsync --progress`, or, for block-level copying, `dd status=progress`. - Use `shuf` to shuffle or select random lines from a file. From 8992f57d5f2ef4534f68e91c4e19e96438da39d0 Mon Sep 17 00:00:00 2001 From: Charles Simchick Date: Mon, 8 Jun 2020 17:09:14 -0700 Subject: [PATCH 2/3] Add info on 'script' and 'scriptreplay' to record and replay terminal session log. --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index 153446c..60b9344 100755 --- a/README.md +++ b/README.md @@ -313,6 +313,12 @@ mkdir empty && rsync -r --delete empty/ some-dir && rmdir some-dir ## System debugging +- To log your session, open a subshell using `script [] [-T ]`. This starts logging of all input and output from your terminal session to `log_file` (`typescript` by default), along with optional timing information to `timing_file`. End the session using `exit`. + +- View the script logging file using `less -r `, which will handle color codes and other raw characters. You can replay a script with timings using `scriptreplay -t `. + +- You can augment your prompt to show when you are recording within a `script` subshell by adding the following to your `.bashrc` file: `lsof -tac script "$(tty)" > /dev/null && PS1="[recording] $PS1"` + - For web debugging, `curl` and `curl -I` are handy, or their `wget` equivalents, or the more modern [`httpie`](https://github.com/jkbrzt/httpie). - To know current cpu/disk status, the classic tools are `top` (or the better `htop`), `iostat`, and `iotop`. Use `iostat -mxz 15` for basic CPU and detailed per-partition disk stats and performance insight. From 7bd0b145d43062472a72da3a53cafe935e29a2c8 Mon Sep 17 00:00:00 2001 From: Charles Simchick Date: Mon, 8 Jun 2020 17:22:46 -0700 Subject: [PATCH 3/3] Add reference to 'bats' for shell script testing suite. --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 60b9344..871ab14 100755 --- a/README.md +++ b/README.md @@ -616,6 +616,7 @@ These items are relevant *only* on Windows. - [awesome-shell](https://github.com/alebcay/awesome-shell): A curated list of shell tools and resources. - [awesome-osx-command-line](https://github.com/herrbischoff/awesome-osx-command-line): A more in-depth guide for the macOS command line. - [Strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/) for writing better shell scripts. +- [bats](https://github.com/sstephenson/bats): A test suite for bash scripts. - [shellcheck](https://github.com/koalaman/shellcheck): A shell script static analysis tool. Essentially, lint for bash/sh/zsh. - [Filenames and Pathnames in Shell](http://www.dwheeler.com/essays/filenames-in-shell.html): The sadly complex minutiae on how to handle filenames correctly in shell scripts. - [Data Science at the Command Line](http://datascienceatthecommandline.com/#tools): More commands and tools helpful for doing data science, from the book of the same name