'Compressing' CloudFormation template to get around size limit

Recently I hit the 51,200 byte body size limit of AWS's CloudFormation's templates.  I looked into creating Nested Stacks, but that seemed like a pain.  Looking at the created json template, I saw a lot of unneeded whitespace.

I used troposhere to generate the template, so it was easy to reduce the size by stripping out the beginning and ending whitespace of each line in the json file.

I just added the following line:
json_compressed="\n".join([line.strip() for line in t.to_json().split("\n")])
utils.validate_cloudformation_template(json_compressed)

This more than havled the size of the template from ~60K to ~25K bytes:

$ wc template.json
    1821    2860   60133 template.json
$ wc template_compressed.json
    1821    2860   24616 template_compressed.json

And CloudFormation accepted it, no problem.

Elixir Tgraph

In my attempt to learn Elixir, I've converted an Erlang version of a Python script I wrote years ago.  It takes a pipe of numeric values and plots lines in a character terminal.  Super simple, but handy sometimes.

Now, I know line count is a horrible way to compare code, but here it is:

150 tgraph.py https://gist.github.com/dgulino/4750099
136 tgraph.escript https://gist.github.com/dgulino/4750118
106 tgraph.ex https://gist.github.com/dgulino/298516f7977c57199a4a

The python code is many years old.  Maybe I've learned something since then, but I don't write very compactly, on purpose.  I only use standard libraries since I don't have the luxury of installing stuff on some of the systems I want to run this on, so can't use some of the cool libraries out there.

It's hard to compare the python code to the Erlang/Elixir.

The Erlang code has added cruft on top to run as Escript (so I don't have to compile changes).   I've yet to figure how to enable piping of stdin to Elixir without running a build ('mix escript.build').  So I get to compile my interpreted code!

Otherwise, the Elixir code is easier to read and more concise than Erlang, which is no suprise.  Elixir +1.

Number of New Connection Per Second

Under pressure, debuging a production system, given the following question:

"How many new connections per second are we creating to S3?".

My one-liner (Linux):

while true; do diff   <(netstat -an | grep ESTAB | grep ":443 "| grep -v "N.N.N.N:443 " | sort) <(sleep 1; netstat -an | grep ESTAB | grep ":443 "| grep -v "N.N.N.N:443 " | sort) | grep "<" | wc -l;sleep 1;done

Where N.N.N.N is the local IP.  Many better ways to do this, but I had this in a few minutes.

Done.

linux mint privacy/security setup

Privacy/Security

Linux(Mint)
dns:
dnscrypt
init.d script
apt-get install sysv-rc-conf
sysv-rc-conf dnsycrypt-proxy on
apt-get install unbound
/etc/unbound/unbound.conf:
forward-zone:
  name: "."
  forward-addr: 127.0.1.1@53
airplane init.d # cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1

# OpenDNS Fallback (configured by Linux Mint in /etc/resolvconf/resolv.conf.d/tail).
nameserver 208.67.222.222
nameserver 208.67.220.220

disable dnsmasq (not caching):
airplane init.d # cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile
#dns=dnsmasq

disabled dnssec in unbound:
    # DNSSEC validation using the root trust anchor.
#    auto-trust-anchor-file: "/var/lib/unbound/root.key"





mail:
thunderbird/openpgp
http://www.mailvelope.com/

tor:
tor-browser
non-exit relay:
    bandwidth limit

secure-delete:
sudo apt-get install secure-delete
srm

password management:
passwordmaker

TODO:
filesystem encryption
#############
android:
apg

#############
firefox:
ghostery
adblock plus (disable reasonable ads)

linux mint 15 (cinnamon) on chromebook pixel

linux mint 15 (cinnamon) on chromebook pixel
(Sorry for the info dump, hopefully will fix soon)

setup mint 15 on chromebook

http://www.reddit.com/r/chromeos/comments/1eqsjp/tutorial_how_to_install_any_linux_distro_on_the/

http://www.webupd8.org/2011/12/how-to-enable-mac-os-x-like-natural.html

date/time in panel:
http://www.foragoodstrftime.com/

sudo apt-get install gnome-tweak-tool

sudo sh ./install_firmware_from_alsa_project.sh

sudo add-apt-repository ppa:zedtux/naturalscrolling
sudo apt-get update

sudo apt-get install naturalscrolling
    enable for trackpad, usb mouse
    enable start on login

firefox:
    default full zoom
    ghostery
    adblock plus

trackpad:
    disable left tap
    enable two finder scroll
    small amount of acceleration

windows tiling management:
gTile extension:
http://cinnamon-spices.linuxmint.com/extensions/view/21
https://github.com/shuairan/gTile/commit/3277b72e84407bc70df0dce95e96a7e283587481

screen:
    add brightness applet to panel

    keyboard shorcuts:
        sudo apt-get install xbacklight
        /usr/bin/xbacklight -dec 10
                        -inc 10

suspend fix:
http://blog.brocktice.com/2013/03/09/running-debian-wheezy-7-0-on-the-chromebook-pixel/

/etc/modules:
    tpm_tis force=1 interrupts=0


touchpad:
    disable right lower corner as right click:
    /usr/share/X11/xorg.conf.d/50-synaptics.conf
     31 ##Section "InputClass"
 32 ##        Identifier "Default clickpad buttons"
 33 ##        MatchDriver "synaptics"

 34 ##        Option "SoftButtonAreas" "50% 0 82% 0 0 0 0 0"
 35 ##EndSection

cursor:
 sudo apt-get install oxygen-cursor-theme
 settings...theme..other settings..mouse pointer: oxy-white

power:
    powertop
    arrow over to Tunables
    hit enter on each 'bad', turn it to 'good'
    esc to exit

    change whether plugged in/not
    http://askubuntu.com/questions/112705/how-do-i-make-powertop-changes-permanent
        add output of "sudo powertop --csv=powertop.csv" to battery /etc/pm/power.d/power

disable bluetooth by default:
    Run gksu gedit /etc/rc.local and add this before line with exit 0:
    rfkill block bluetooth

#####################

Cinnamon desktop optimizations:

windows management:
x-tile

keyboard launcher:
kupfer


themes:
window borders: HighContrast
check "show icons on buttons"

cursor:
fix comixcursors
apt-get install comixcursors
get .tar.bz2
extract to:
/etc/X11/cursors/
/usr/share/icons/
reload cinamon alt-f2 "r"


tar xjf ComixCursors-0.7.3.tar.bz2

monkeyrunner hanging on input/raw_input on Mac OS X: RESOLVED

I found recently that my monkeyrunner scripts started failing after updating the Android SDK Tools to version 22.0.1.  This is a known bug in the jython version shipped with it.  Luckily it's easily fixed by replacing one .jar file.

http://www.jython.org/latest.html:

Jython 2.5.4rc1 Release Notes
Bugs Fixed:

[ 1972 ] jython 2.5.3 sys.stdin.readline() hangs when jython launched as subprocess on Mac OS X

1)  Download latest jython 2.5.4rc1:

http://search.maven.org/remotecontent?filepath=org/python/jython-standalone/2.5.4-rc1/jython-standalone-2.5.4-rc1.jar

2) Copy into ${ANDROID_ROOT}/sdk/tools/lib/

3) move or delete the existing jython-standalone-2.5.3.jar

That's it!  monkeyrunner now process raw_input() and input() correctly.

Move over Yahoo Notepad, welcome Github Gist

Github's gist are a pretty good match for a lot of stuff I do; simple one file scripts, not big projects.

I've been putting up a few scripts I've embedded in this blog, and I'm looking through old files for more.  I even have some stuff in an old Yahoo Notepad.  Yea, Yahoo Notepad.  It was/is a simple way to store text files in folders.  No support for sharing, and you had to cut and paste into a form to 'upload' them.  There doesn't seem to be any links in YMail to that system anymore, so you have to go directly to notepad.yahoo.com; but it still works.  I've had a Yahoo account since they first offered them back in 1997.  Unfortunately, during the early times they had pretty restrictive mailbox size limits (I think it started at 5MB), so I had to delete a bunch of stuff back then.  The earliest mail I still have is dated September 5, 2003.

It's terribly thing relegating some of this stuff to Notepad; I had embedded version numbers in the scripts themselves.  Not quite as reliable as Git, I must say.

My Gists:
https://gist.github.com/dgulino