Github without password

You don't want to type your github password every time you push on your repo. What you need to do is simply to put your public key in github.

1. In the client where you want to clone your git repository just create a private / public key pair:

ssh-keygen -t rsa -b 4096

PS: Leave the keyring password blank



2. Copy your public key (your private will stay in your client) 


cat ~/.ssh/id_rsa.pub

3. Go to the github  SSH  keys settings page  and copy the public key above


4. You should be done! 

logging your screen output in linux bash

Redirecting the bash output does not work with the classic redirection operator > ? No worries, screen comes with a built in option to dump everything when screen is terminating. Just add -L to your screen command as in the example below:

screen -L -S myGreatScreen my_bash_command

Switch your EC2 instance from command line with python and BOTO

First thing first: security. Even before starting messing around with your AWS credentials make yourself sure to get the required precautions.

You dont want to wake up and find that someone has launched 60 c3.x8 servers across 5 regions using the credentials that you left in a backed up directory...

First thing create a user if you dont have yet:  go in the IAM console, create new user and that's it ..

Then the more interesting part: associate a user policy to that user. This will limit the amount of messing that the user can do into your AWS account.

1. click on the user -> in users policies -> click on "Attach User Policy"
2. Click on custom policy generator
3. give your policy a name (e.g. 'my-restricted-policy')
4. copy and paste the following policy 

{

   "Version": "2012-10-17",

   "Statement": [{

      "Effect": "Allow",

      "Action": [

        "ec2:DescribeInstances", "ec2:DescribeImages",

        "ec2:DescribeAvailabilityZones",

        "ec2:StopInstances", "ec2:StartInstances"

      ],

      "Resource": "*"

    }

   ]

}

You are almost done: you just need to generate your security credentials for the user with the limited policy generator. 

So click on the user -> select Security Credentials, click on Manage Access Keys and then Create access key. 

At this point you should have something that looks like the following

aws_access_key_id        = 'DSFSDFSDFWEFEWF'

aws_secret_access_key  = 'ldfjjs8wnoliencdnscdmsfkmsdkfml32'

First part is done, now a few more lines of python and that 's it : 

import boto.ec2


name_of_the_instance = 'put-here-the-name-of-your-instance'
name_of_the_region ='put-here-the-name-of-the-region-where-the-machine-is-located-eg-eu-west-1' 

# access key only for on / off
aws_access_key_id_     = 'DSFSDFSDFWEFEWF'
aws_secret_access_key_ = 'ldfjjs8wnoliencdnscdmsfkmsdkfml32'


conn = boto.ec2.connect_to_region(region_name, 
                                  aws_access_key_id     = aws_access_key_id_,
                                  aws_secret_access_key = aws_secret_access_key_ )

inst = conn.get_all_instances(filters={'tag:Name': name_of_the_instance})[0].instances[0]
print inst.stop()

handle csv in just one line with bash

gawk -vFPAT='[^,]*|"[^"]*"' '{print $6}' `

screen and matlab ( chronicles of an hour wasted.. )

When you are executing matlab in background on a server by screening the command you obviously cannot use the matlab interface. To disable it just use the -nodisplay option.

matlab -nodisplay -nosplash  -r  "download('$(LIST_)')"

Be careful, dont even attempt to use the -nodesktop option because your screen command will never work. The explanation is in this post And this is an alternative to launch your screen in background

screen -A -m -d -S  matlab matlab -nodisplay -nosplash

Install matlab in an EC2 instance

I assume you have an instance and an attached EBS

1. Upload your copy of matlab to your instance with rsync:

 rsync -avL --progress -e "ssh -i .ssh/.pem"\
 matlab_installation ubuntu@ 

2. Edit the file installer_input.txt and set the follwing options:

## Set mode value to either interactive, silent, 
## or automated and
## uncomment the line.

mode= silent


## Set the desired value for destinationFolder and 
## uncomment the line. 

destinationFolder=/usr/local/MATLAB/R2012a/

## Set agreeToLicense value to yes or no and 
## uncomment the line.

agreeToLicense=yes

## Set the desired value for outputFile and 

## uncomment the line.

outputFile=/tmp/mathworks_luca.log

3. Set the following option in your ~/.bashrc

emacs -nw ~/.bashrc 
export LC_ALL="en_US.utf8" 

4. Install Java Runtime Environment

ubuntu:

sudo apt-get install default-jre

centos

sudo yum install java7

5. Install extra libraries

centos

sudo yum install libXmu.x86_64
sudo yum install libXt.x86_64
sudo yum install libXp.x86_64

NOTE:  you might need the 32 bits version if you are installing into a 32bits architecture

5. Run the installation file

sudo ./install -inputFile installer_input.txt

6. Set your path

..if bash is your favorite shell edit your .bashrc file

emacs -nw ~/.bashrc

insert somewhere: 

export PATH=/usr/local/MATLAB/R2012a/bin:$PATH

or  

alias matlab='/usr/local/MATLAB/R2012a/bin/matlab -nosplash -nodesktop -c /usr/local/MATLAB/R2012a/lic_standalone.dat'

..if fish is your favourite shell edit your .config file

emacs -nw  ~/.config/fish/config.fish

# insert somewhere 

set -g -x PATH /usr/local/MATLAB/R2012a/bin $PATH
 

 7. Copy the license :

cp /wherever/it/is/my/lic_standalone.dat /usr/local/MATLAB/R2012a/lic_standalone.dat

mounting devices

mounting an iso:

sudo mount -o loop Office_ProPlus_2013_VL_32Bit_En-US-Sep2013.iso  iso/

sudo mount -o loop ml2012au.iso  /media/matlab

mounting an iso:

sshfs -o idmap=user $USER@:/media/source ~/destination

Create a tar archive split into blocks of a maximum size

Let's say you have a lot of data and instead of a big tar file you want smaller ones.
Let's say that you want 200MB files. Very simple, just type

tar cvzf - dir/ | split --bytes=200MB - sda1.backup.tar.gz

NOTE that to untar you have to cat the files before:

cat sda1.backup.tar.gz.* | tar xzvf 

Credits to Jordanm for post

all you ned to know about matlab from command line

1. create your function

function download(file_input)
path_root = './';
mkdir(path_root);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% INPUT 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%% file  input
if length(file_input)<1 font="">
    exit(1);
end

.....

2. save it in a file called with the same name of the function

download.m

3. run matlab with the following command

$ matlab -nodesktop -nosplash  -r  "download('')"

Convert iTunes playlists in clementine / linux .m3u

1. Export one by one all your iTunes playlists in a folder called iTunes_playlists/

2. Examine one of them. In my case I have the music stored in the Z:\\ folder

#EXTM3U
#EXTINF:329,Thank You / Rebel And A Cause - Just A Band
Z:\music\iTunes\iTunes Media\Music\Just A Band\Sorry For The Delay\03 Thank You _ Rebel And A Cause.mp3
#EXTINF:216,Till We Try - Just A Band
Z:\music\iTunes\iTunes Media\Music\Just A Band\Sorry For The Delay\05 Till We Try.mp3


3. In my case I have to get rid of all the Z:\ and of all the backslashes \ to be linux compatible. 


#!/bin/bash
mkdir -p  music/playlists/; for i in  iTunes_playlists/*.m3u; do sed -e 's@Z:\\@/home/<your_username>/@' -e 's@\\@/@g' ${i} > music/playlists/${i}; done

Histogram of your Evernote tags in one line

1. Export all your notes in an enex file:

/home/luca/tmp/bridge/all.enex

Run the following line

echo -e "import re\nfor tag in re.findall('.*?',open('/home/luca/tmp/bridge/all.enex').read(),re.DOTALL):print tag" | python | sed -e 's@@@' -e 's@@@' | sort | uniq -c | sort -rg | m

To read a file in a loop

To read the file line by line use:

cat sample.txt | while read line; do echo $line; done 
To read the file element (word) by element you can use:


for word in `cat sample.txt`; do echo $word; done

or 

while read line; do echo $line; done  <  sample.txt

credit +Isaac Alonso 

greedy vs non greedy pattern in sed

sed -e 's@^.*/@@' -e 's@_.*@@' ../learning/lists/faces.encl

sed -e 's@^.*+/@@' -e 's@_.*@@' ../learning/lists/faces.encl

crontab tweakings

Remember to place a minimal configuration in your /etc/crontab or in your crontab -e

MAILTO=luca.marchesotti@gmail.com

SHELL=/bin/bash

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:~/bin

Exec format error (in cron)

If you are like me, you sometimes get lazy and forget to include the shell command at the top of your scripts. This usually isn't a problem, but in certain cases (where the shell isn't set, or doesn't exist yet) it will cause problems. So, even if your script is set to be executable (chmod +x), you'll receive an error like:

Exec format error

Just such a case manifests itself when using a script through cron (and run-parts). To remedy this problem put the following at the beginning of the script:

#!/bin/sh




Credit: 

Brian Oneill

Difference between quote and double quote on bash

sed -e '/^$$/d' 
sed -e "/^$$/d" 
`
echo '$${c}'
echo "$${c}"

awk  wants the single quote 

when you use single quotes the variables aren’t going to be expanded.http://www.howtogeek.com/howto/29980/whats-the-difference-between-single-and-double-quotes-in-the-bash-shell/

using regexp in bash

Using regexp in bash can simplify your life (and the visualization of list of files..). Have a look to this:

$> ls sgd[^_]*

all files which do that begin with sgd and DO NOT CONTAIN the pattern _

credit +Isaac Alonso 

remove duplicate words in lines

simple line that *should* remove duplicate words from a text file by looking at each line independently:

while read l; do echo ${l} | tr ' ' '\n' | sort  | uniq  | tr '\n' ' ';echo ""; done < lists/challenges_test_pos_contains_duplicates.labl > lists/challenges_test_pos.labl



NOTE WELL: test it before using it

when you cannot give as input a list of file ...

Command like montage dont  take a list of files as an input argument. What to do? simple, just open your list of files..

in bash this is easy:


montage `cat tmp.txt` -resize 150x150 -crop 150x150+0+0 -gravity center   -tile 8x8 -geometry +7+7 -page A4  filename.jpg


within a makefile rule it becomes a bit more complicated, you have to echo a variable where you cat the file

montage `echo $$(cat tmp.txt)` -resize 150x150 -crop 150x150+0+0 -gravity center   -tile 8x8 -geometry +7+7 -page A4  $@

Redirect bash stdoutput

use tee!! 

from wikipedia :

colorful bash and makefile output

enhance readability of your makefile output by using colormake! :

1. install colormake in your favorite directory (e.g. ~/bin)
2. make an alias to the executable in your ~/.bashrc

alias make='~/bin/colormake-0.9/colormake'


enjoy the colors!

in addition to have consistent colors in emacs over different bashes export

export TERM=xterm-256color