Numeric file permissions help

Sorry for the delay in posting another howto.  I have been working on a set of network knocking scripts (server daemon, init.d, and client).  I should be posting them soon. For now though...

I find when working on scripts that remembering numeric representations of file permissions can be a pain. So I wrote a quick 1liner to paste and run, to help decide.

It creates a temp-file in the current directory, loops through changing the permissions, and prints the current number, along with what permissions the user has with that number represented. You can then use that to decide which number to use for each (user|group|other). Finally it removes the temp-file again.

There are tables out already, but I wanted a quick function in terminal for myself. so I include the following function in my default functions-rc file. Here is the function:

fileperms(){ testfile=$(tempfile -d ./);for i in {1..7};do chmod 0$(($i))00 $testfile;echo "$i $(ls -l $testfile |cut -d ' ' -f1)";done;rm $testfile ; }

You can then use it by just calling 'fileperms'

Hope you find it useful and thanks for reading.

tscp - Script to quickly and securely transfer files by tunneling tar through ssh

For security, pretty much every file I transfer over the network I use 'scp'.  Or more likely (to save time and traffic) I tunnel 'tar' through 'ssh'.  This compresses the data, transferes securly through the network, and then de-compress on the other end.  But it can be a pain remembering the exact syntax, and others have requested a script to help with this.

So to help I wrote the following script:


#!/bin/bash
#
#       tscp
#
#       Copyright 2011 nairb <code@nairb.us>
#
#       This program is free software; you can redistribute it and/or modify
#       it under the terms of the GNU General Public License as published by
#       the Free Software Foundation; either version 2 of the License, or
#       (at your option) any later version.
#
#       This program is distributed in the hope that it will be useful,
#       but WITHOUT ANY WARRANTY; without even the implied warranty of
#       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#       GNU General Public License for more details.
#
#       You should have received a copy of the GNU General Public License
#       along with this program; if not, write to the Free Software
#       Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
#       MA 02110-1301, USA.

tarscp(){
  if [ $1 ] ; then
    myaddress=''
    localfile=''
    localtoremote=''
    if [ -z sshopts ]; then
      sshopts="-2 -A"
    fi
    for i in $@ ; do
      if [ $(echo $i |grep -iE '^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}:.*') ] ; then
        myaddress=$i
        if [ $localfile ] ; then
          localtoremote="true"
        fi
      else
        if [ -f $i -o -d $i ] ; then
          localfile=$i
        fi
      fi
    done
    if [ $myaddress ] ; then
      remotepath=$(echo $myaddress |cut -d ':' -f2)
      myaddress=$(echo $myaddress |cut -d ':' -f1)
    else
      echo -e "yo, come on...\nneed to give a full address and path\nlike: 'myuser@myhost.com:~/some/folder/or/file'" && return 1
    fi
    checkremotepathcom="if [ -d $remotepath -o -f $remotepath ] ; then echo 0 ; else echo 1 ; fi"
    if [ $(ssh $sshopts $myaddress "$checkremotepathcom") == 0 ] ; then
      if [ $localfile ] ; then
        if [ $localtoremote ] ; then
          checkremotepathcom="if [ -d $remotepath ] ; then echo 0 ; else echo 1 ; fi"
          if [ $(ssh $sshopts $myaddress "$checkremotepathcom") == 0 ] ; then
            remotecom="cd $remotepath ; tar -xzf -"
            tar -czvf - $localfile |ssh $sshopts $myaddress "$remotecom"
          else
            echo "yo dude. that path aint on the server." && return 1
          fi
        else


          if [ $remotepath == '~/' ] ; then

            remotefile=$(ssh $sshopts $myaddress "pwd")

            remotefile=$(echo $remotefile |cut -d '/' -f $(echo $remotefile |sed -e 's/\//\/ /g' |wc -w))

            remotecom="cd $remotepath ; cd ../ ; tar -czvf - $remotefile"

          else

            remotefile=$(echo $remotepath |cut -d '/' -f $(echo $remotepath |sed -e 's/\//\/ /g' |wc -w))

            if [ -z $(echo $remotepath |cut -d '/' -f 1) ] ; then

              mypath='/'

            else

              mypath='~/'

            fi

            for i in $(seq 2 $(($(echo $remotepath |sed -e 's/\//\/ /g' |wc -w) - 1))) ; do

              mypath=$mypath/$(echo $remotepath | cut -d '/' -f $i)

            done

            remotepath=$mypath

            mypath=''

            remotecom="cd $remotepath ; tar -czvf - $remotefile"

          fi

          if [ -d $localfile ] ; then

            cd $localfile

          fi

          if [ $remotepath == '/' ] ; then

            remotecom="tar -czvf - /"          

          fi

          ssh $sshopts $myaddress "$remotecom" | tar -xzf -

        fi

      else

        if [ $remotepath == '~/' ] ; then

          remotefile=$(ssh $sshopts $myaddress "pwd")

          remotefile=$(echo $remotefile |cut -d '/' -f $(echo $remotefile |sed -e 's/\//\/ /g' |wc -w))

          remotecom="cd $remotepath ; cd ../ ; tar -czvf - $remotefile"

        else

          remotefile=$(echo $remotepath |cut -d '/' -f $(echo $remotepath |sed -e 's/\//\/ /g' |wc -w))

          if [ -z $(echo $remotepath |cut -d '/' -f 1) ] ; then

            mypath='/'

          else

            mypath='~/'

          fi

          for i in $(seq 2 $(($(echo $remotepath |sed -e 's/\//\/ /g' |wc -w) - 1))) ; do

            mypath=$mypath/$(echo $remotepath | cut -d '/' -f $i)

          done

          remotepath=$mypath

          mypath=''

          remotecom="cd $remotepath ; tar -czvf - $remotefile"

        fi

        if [ $remotepath == '/' ] ; then

          remotecom="tar -czvf - /"          

        fi

        ssh $sshopts $myaddress "$remotecom" | tar -xzf -

      fi

    else

      echo "dude, that path aint on the server." && return 1

    fi

  else

    echo -e "no arguments given fool\ntry again" && return 1

  fi

}




case $(echo ${0##*/} |cut -d '/' -f $(echo ${0##*/} |wc -w)) in

  tscp)

    tarscp $@

    ;;

  *)

    echo -e "come on... just call the script 'tscp'"

    ;;

esac

And, if like me, you find yourself logging into a server that doesnt have the script saved, and you dont want to make a new script file (maybe you only need to use the command once to migrate data from your old server to a new one). I converted the above into a 1-liner function that can just be pasted in and used with the same call.  Alternatively you could include this function in one of your functions rc files like mentioned in my previous post.

Regardless of the reason, here is the 1-liner function:


tscp(){ if [ $1 ] ; then myaddress=''; localfile=''; localtoremote=''; if [ -z sshopts ]; then sshopts="-2 -A"; fi; for i in $@ ; do if [ $(echo $i |grep -iE '^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}:.*') ] ; then myaddress=$i; if [ $localfile ] ; then localtoremote="true"; fi; else if [ -f $i -o -d $i ] ; then localfile=$i; fi; fi; done; if [ $myaddress ] ; then remotepath=$(echo $myaddress |cut -d ':' -f2); myaddress=$(echo $myaddress |cut -d ':' -f1); else echo -e "yo, come on...\nneed to give a full address and path\nlike: 'myuser@myhost.com:~/some/folder/or/file'" && return 1 ; fi; checkremotepathcom="if [ -d $remotepath -o -f $remotepath ] ; then echo 0 ; else echo 1 ; fi"; if [ $(ssh $sshopts $myaddress "$checkremotepathcom") == 0 ] ; then if [ $localfile ] ; then if [ $localtoremote ] ; then checkremotepathcom="if [ -d $remotepath ] ; then echo 0 ; else echo 1 ; fi"; if [ $(ssh $sshopts $myaddress "$checkremotepathcom") == 0 ] ; then remotecom="cd $remotepath ; tar -xzf -"; tar -czvf - $localfile |ssh $sshopts $myaddress "$remotecom"; else echo "yo dude. that path aint on the server." && return 1 ; fi; else if [ $remotepath == '~/' ] ; then remotefile=$(ssh $sshopts $myaddress "pwd"); remotefile=$(echo $remotefile |cut -d '/' -f $(echo $remotefile |sed -e 's/\//\/ /g' |wc -w)); remotecom="cd $remotepath ; cd ../ ; tar -czvf - $remotefile"; else remotefile=$(echo $remotepath |cut -d '/' -f $(echo $remotepath |sed -e 's/\//\/ /g' |wc -w)); if [ -z $(echo $remotepath |cut -d '/' -f 1) ] ; then mypath='/'; else mypath='~/'; fi; for i in $(seq 2 $(($(echo $remotepath |sed -e 's/\//\/ /g' |wc -w) - 1))) ; do mypath=$mypath/$(echo $remotepath | cut -d '/' -f $i); done; remotepath=$mypath; mypath=''; remotecom="cd $remotepath ; tar -czvf - $remotefile"; fi; if [ -d $localfile ] ; then cd $localfile; fi; if [ $remotepath == '/' ] ; then remotecom="tar -czvf - /"          ; fi; ssh $sshopts $myaddress "$remotecom" | tar -xzf -; fi; else if [ $remotepath == '~/' ] ; then remotefile=$(ssh $sshopts $myaddress "pwd"); remotefile=$(echo $remotefile |cut -d '/' -f $(echo $remotefile |sed -e 's/\//\/ /g' |wc -w)); remotecom="cd $remotepath ; cd ../ ; tar -czvf - $remotefile"; else remotefile=$(echo $remotepath |cut -d '/' -f $(echo $remotepath |sed -e 's/\//\/ /g' |wc -w)); if [ -z $(echo $remotepath |cut -d '/' -f 1) ] ; then mypath='/'; else mypath='~/'; fi; for i in $(seq 2 $(($(echo $remotepath |sed -e 's/\//\/ /g' |wc -w) - 1))) ; do mypath=$mypath/$(echo $remotepath | cut -d '/' -f $i); done; remotepath=$mypath; mypath=''; remotecom="cd $remotepath ; tar -czvf - $remotefile"; fi; if [ $remotepath == '/' ] ; then remotecom="tar -czvf - /"          ; fi; ssh $sshopts $myaddress "$remotecom" | tar -xzf -; fi; else echo "dude, that path aint on the server." && return 1 ; fi; else echo -e "no arguments given fool\ntry again" && return 1 ; fi; }


Hope you find this helpful. Thanks for reading.


***** UPDATE *****
Sorry to anyone who copied this earlier today.  The functionality worked for some folders (not including '~/', or '/'), but it has now been edited to work for those. Also now including ssh options like '-2 -A' and I have fixed it to work for single file, which failed before (since you can only 'cd' into a directory, not a file)

Also the prior 'exit' lines have been changed to 'return' to prevent the terminal window being closed if you run from the function without the right arguments.

Again sorry for posting it prior to thorough testing (these things hapen with insomnia posting scripts at 5am with no sleep), but it has been fixed now, and tested on both debian, and ubuntu computers/servers

I may still update this again to make it more organized, but will likely release future revisions as file downloads on my own site instead of this blog (its a kinda long script already to copy and paste)
****

HTML Safe Characters - Bash function using 'sed'

I find myself posting the output from commands in terminal to html pages often, so wrote a quick "1-liner" bash function to convert a string to (mostly)html safe characters. you can "pipe" a string to this function, and direct its output to another file.

The function:

tohtm(){ IFSBACK=$IFS;IFS=$(echo -en "\n\b");string2htm(){ [[ $1 ]] && echo $1|sed -e 's/\&/\&amp;/g' -e 's/\"/\&quot;/g' -e 's/</\&lt;/g' -e 's/>/\&gt;/g' -e 's/>/\&gt;/g'; };if [ ! -t 0 ];then echo -e "<html>\n<head></head>\n<body>\n<p>";for i in $(cat /dev/stdin); do string2htm $i;echo "<br/>";done;echo -e "</p>\n</body>\n</html>";fi;IFS=$IFSBACK; }

Syntax to run this function would be is similar following example commands:

ls - la |tohtm >thisfolder.html

Or

echo 'this & that is < than "something", but > than "something else"' |tohtm

Both of the above commands should work, but there may be others that don't (if the string includes single quotes for example). I still need to do some work on initially formatting such strings, but this function has been very useful for me so far.

you can also simply use cgi.escape from python. for example:

2htm(){ if ! [ -t 0 ];then myvar=$(cat /dev/stdin); else if [ "$1" ];then myvar=$@;fi;fi;pycom=$(echo -e "from cgi import escape\nprint escape(u'''$myvar''')"); python -c "$pycom"; }

That is about all I have for this post. I will post other similar functions (and perhaps this one again) in a future post.  Thanks for reading.

User owned scripts security

Since I am posting scripts (bash, pythong, ect.) up here, I figure it would be a good idea to mention security and permissions of these (or any of you own/other's) scripts.  If you make a script executable (chmod +x scriptname), Linux can use the shebang to determine the script type, and run with a call similar to any other application on the system.  However, if like me you care about security, you may not want to have executable files in your home directory.  One of the best things about linux (or any *nix including bsd, or mac) is the user permissions management.  If your account gets compromised, that doesn't mean the rest of your whole system has been "rooted" (compromised as the root user).  Most infections (linux is not immune to compromise) attack executable files that your user has access to, and so if you have executable scripts in your home directory, they can be written to.  Then the next time you call on one of these scripts, you may be running a malicious hacker's code.

Now you may ask “what alternative is there?”.  With the rest of this post, I plan to give a few.

root owned (standard for most systems):
If it is a script that others on the system would want to use (and you can become root), you could move it into one of the system folder in the users $PATH.  I don't tend to do this often (and dont really suggest it), since it puts random scripts outside of the system's package management.  Instead it may be best to use one (or a combination like I do) of the following options. You can then give the other users the file (use links, edit their rc files (~/.bashrc, ~/.profile) as root, or include them in /etc/skell before creating the users) .

rc file:
As stated in my previous post, you may create a file with a list of simple functions. If its a simple script, it may be fairly easy to convert to a function (or a few).  If you can create a few functions you would want to use together, include them into an 'rc' file.  Then when you want to use them, you only need to 'source' the file first (source MyFunctions.rc). You can then call on the functions just like any other command. If it is a list of functions you would want to use (or at least be able to without sourcing the file) every time you login, you could include sourcing the rc file through your ~/.bashrc (or ~/.profile if you might want to use these functions as shortcuts in your GUI shell such as gnome. I may go into this further in another post, but thats beyond the scope of this article).

Not every script can easily be converted to the function, and so

alias the complex scripts:
most modern linux distrobutions include a tag in your ~/.bashrc, or .profile to source a ~/.bash_aliases file.  Create, and use this file if you dont already have it. And a statement similar to the following 'if' should be included your ~/.bashrc, or ~/.profile

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

!!you can use a this as example of sourcing to source one of the previous mentioned functions rc files in your bashrc, or profile.

Create your ~/.bash_aliases file and fill it with statements similar to the following example to create a command to run a non-executable bash script called ~/myscripts/newscript as a command 'myscript'

alias myscript=”bash ~/myscripts/newscript”

your ~/.bashrc file is parsed every time you login to bash, your .profile is sourced every login (including most GUI shells such as gnome, kde, xfce4) so if you have non-executable scripts that use a gui toolkit (such as zenity for gnome) you may want to source another ~/.gui-aliases or similar in your ~/.profile so you can easily run said aliased script through terminal, or a shortcut in your applications menu, panel, or desktop.

Thats about all for this post. I will probably give an example rc file (with some fun included functions) that auto-aliases all scripts under a folder based on the “shebang” line in a future post.