Apr 30 2007

Sed reminders and tricks

Tag:tepezcuintle @ 21:38

More tips

here is another reminder of how to fix certain files

say you have a file that has lines and therere are blank
lines and you need to remove them.

sed ‘/^$/d’ original.txt > newfile.txt

so basically that will remove all blank lines from a file.
Very cool indeed.

let’s say you run the one liner above and you still can’t remove
the blank lines. There must be some leading blank spaces you can’t see

to remove them you have to run this one liner first.

sed ’s/^[ \t]*//’ original.txt > leadingspacegone.txt

then run the onle liner to remove the blank lines and you will be set.

More one liners at.

http://soft.zoneo.net/Linux/remove_empty_lines.php

http://sed.sourceforge.net/sed1line.txt

#######################################################

I am using sed to clean up some code for an AIM robot that I am developing. here are some strings I am using.


How to remove extra spaces from a file .

sed -e 's/\([^ ]\) \+/\1 /g’ < users.txt > newusers.txt

here the file is piped into the sed command and then redirected to newusers.txt
this will remove all the extra spaces on the file.

This is sed one liner that adds a leading
to each line.

sed ’s/^/
/’ < users.txt > users.txt.spaced the file is read
from users.txt and then piped into users.txt.spaced.

Continua”Sed reminders and tricks”


Apr 26 2007

Working with temporary files using the Shell

Tag:tepezcuintle @ 0:22

The basic idea of a symlink exploit is to predict where an application will create its temporary file and put a symbolic link (symlink) at that place. When the application now tries to work with a tempfile it will actually work with the file that the symlink points to. The most simplistic thing a (local) attacker can do with this is to have the symlink point to important files such as /bin/bash. This will cause the program (assuming it has root privileges) to overwrite /bin/bash with its temporary data, making the system unusable. . . .
Safely Creating Temporary Files in Shell Scripts

Continua”Working with temporary files using the Shell”


Apr 25 2007

How to encrypt Shell scripts

Tag:tepezcuintle @ 16:48

Do you have scripts that contain sensitive information like passwords and you pretty much depend on file permissions to keep it secure? If so, then that type of security is good provided you keep your system secure and some user doesn’t have a “ps -ef” loop running in an attempt to capture that sensitive info (though some applications mask passwords in “ps” output). There is a program called “shc” that can be used to add an extra layer of security to those shell scripts. SHC will encrypt shell scripts using RC4 and make an executable binary out of the shell script and run it as a normal shell script. This utility is great for programs that require a password to either encrypt, decrypt, or require a password that can be passed to a command line argument.
Download shc (http://www.datsi.fi.upm.es/~frosal/) and untar it:

tar -xzvf shc-X.X.tgz
cd shc-X.X/
make
make install

A binary named “shc” will be created along with some test programs. Let’s give it a try.

Create a file called: “script.sh” and add the following contents:

############################### script.sh ##############################
#!/bin/sh

echo “I love Duane’s articles and will send him a donation via PayPal.”

############################### script.sh ##############################

Now run the command:

shc -f script.sh

The switch “-f” specifies the source script to encrypt. The above command will create two files: script.sh.x.c and script.sh.x.

The program “shc” creates C source code out of your shell script then encrypts it (script.sh.x.c). The encrypted shell script is: script.sh.x. Run that binary and see the output:

./script.sh.x
I love Duane’s articles and will send him a donation via PayPal.

Now copy the original “script.sh” file to a floppy disk or some other system for backup or in case you need to edit it in the future. Then, delete it from the server and delete the “script.sh.x.c” file it creates.

Neat feature

You can also specify a time limit on the shell script so that it will no longer execute after a certain date and you can specify a custom message to echo back to the user. Run this command on the “script.sh” file we created earlier in this tut:

shc -e 09/10/2004 -m “Dude it is too late to run this script.” -f script.sh
./script.sh.x
./script.sh.x has expired!
Dude it is too late to run this script.

In the above command the date October 9, 2004 is set as the expiration date (-e 09/10/2004) and the custom message was set to display to the user (-m “Dude it is too late to run this script.”) when the binary is executed. Note the date format is dd/mm/yyyy.

Check out the man pages for more info on “shc”. Remember that the binary is only encrypted on the local system. If you encrypt a script that transmits sensitive information in clear text across a network, you will need some other encrypted communication channel to transmit that information.


Apr 25 2007

Bash passing arguments with getops

Tag:tepezcuintle @ 16:38

Today I was writing some scripts, and in every script I wanted something to handle all input arguments, in a good way, so I could pass my arguments in any order and my program would know about it.

I used ‘getopts’ before, but this time I decided to write some stuff here about it.

Let me show you how useful it can be:

Let’s suppose that I’m writing a test script, that needs, as argument, the type of the test, the server, the server root password and for debugging purpose we’re going to have a verbose flag too. So, putting it down:

  • “-t” - the type of the test, let’s suppose we have “test1″ and “test2″
  • “-s” - the server
  • “-p” - the root password of the server
  • “-v”- a flag just to let the script run in a verbose mode

Ok, now how we’re going to write this script and parse these arguments? We can use the harder way, fixing an order and parsing it by hand at the script, something like this:

salveti@evalap /tmp/scripts $ cat test_script.sh
#!/bin/bash
# Argument order = -t test -r server -p password -v
TEST=$2
SERVER=$4
PASSWD=$6
if [[ $# -gt 6 ]]
then
    VERBOSE=1
else
     VERBOSE=2
fi

Alright, this works, but if you want to run the script with the arguments in a different way? Or if you forget and put it in the right order? It’ll not work, so, this is an ugly solution.

Ok, but how can you deal with arguments not worrying about the order and if needs an argument or not? Getopts is the answer ;)

Let’s see how we can write the script using getopts and them we explain how it works.

The new script (it’s bigger, I’ll explain why):
#!/bin/bash
# Argument = -t test -r server -p password -v

usage()
{
cat << EOF
usage: $0 options

This script run the test1 or test2 over a machine.

OPTIONS:
   -h      Show this message
   -t      Test type, can be ‘test1′ or ‘test2′
   -r      Server address
   -p      Server root password
   -v      Verbose
EOF
}

TEST=
SERVER=
PASSWD=
VERBOSE=
while getopts “ht:r:p:v” OPTION
do
     case $OPTION in
         h)
             usage
             exit 1
             ;;
         t)
             TEST=$OPTARG
             ;;
         r)
             SERVER=$OPTARG
             ;;
         p)
             PASSWD=$OPTARG
             ;;
         v)
             VERBOSE=1
             ;;
         ?)
             usage
             exit
             ;;
     esac
done

if [[ -z $TEST ]] || [[ -z $SERVER ]] || [[ -z $PASSWD ]]
then
     usage
     exit 1
fi

In this script I created a usage function, just to help you explaining all arguments.

Then, we can see the getopts’ call while getopts "ht:r:p:v" OPTION, this is the main point of the script, it’s how we deal with arguments using getopts. Getopts require an optstring and a var name, just to help you checking the arguments.

When you call getopts, it will walk in your optstring argument, identifying which argument needs a value and which don’t. After getting an argument, getopts set the OPTION var, so you can check it using a case code block, or something like that. If your argument needs a value, getopts will set the var $OPTARG with the value, so you can check and see if it’s what you were expecting (in this example, check if the test argument is passed with “test1″ or “test2″). Easy hã?

Ok, but what is this “:” doing in the arguments? And why the arguments “h” and “t” are together?

This is an import point of getopts. You can use “:” in two cases, one when you want getopts to deal with argument’s errors, and another to tell getopts which argument needs a value.

First, the error checking. When you pass the arguments to getopts in the optstring, getopts will only check what’s there, so if you pass an argument that’s not listed at optstring getopts will give an error (because it’s not a valid argument). When you put “:” at the beginning of the optstring, “:ht:r:p:v” for example, getopts sets the OPTION var with “?” and the $OPTARG with the wrong character, but no output will be written to standard error; otherwise, the shell variable $OPTARG will be unset and a diagnostic message will be written to standard error (./test_script.sh: illegal option — l, if you pass the argument -l, for example).

Second, how to tell getopts which argument needs a value. When you need an argument that needs a value, “-t test1″ for example, you put the “:” right after the argument in the optstring. If your var is just a flag, withou any additional argument, just leave the var, without the “:” following it.

So, in the example, you can see that I’m leaving the error checking to getopts, the vars “t”, “r”, “p” needs a value and “v” is just a flag.

To finish the script, we have a var checking, just to see if all vars that needs a value are not empty.

And, that’s it. For now, you can try making a new script and playing with it a little, it’s not so hard and can help you very much when writing new scripts :)


Apr 17 2007

Breaking my linux server while trying to get open ldap installed

Tag:tepezcuintle @ 17:54

So this weekend. I was trying to install openldap the most recent version on my server at home. So I did it and like always when you try to install some new software on your computer the configure script usually fails saying you have some missing or outdated libraries.

I ran the configure script for my ldap source code and it failed saying that I had an older version of cyrus-sasl so I went and downloaded the cyrus-sasl software and installed a new version. The new version got installed an then I was able to configure my openldap build. I ran make install and I was able to install openldap. I was really happy because it was working and I was able to build an address book and get other cool information on my new ldap server database.

So guess what a day later I tried to open pine to check one of my accounts and it said that some libldap.so file was missing , oh no this sucks.
It was trying to look for a library so what I did I setup a symlink to point to a new libldap.so that was installed by the new ldap software.

So after that got fixed I got another library error message when trying to load pine. So fine I had to create another symlink to another library that was installed by the new ldap software.

Once i fixed that symlink I was able to start the pine software. Cool then I noticed that I was having problems with sending mail saying that my outgoing email username and password was not working. I restarted sendmail and guess what I got an error message saying I had some missing
cyrus-sasl libraries missing. Damn

It looks like everything got hosed, this was a nightmare. I don’t want to make things longer but let me say that I screwed up my server, usually when I build software I install it using the make install command and i choose the folder where things should be installed.

My mistake was that I used checkinstall to install this new software. Checkinstall basically creates rpms and installs them and what happened is that after I built the software and installed it checkinstall just overwrote my original RPM’s .

I should’ve just done make install because make install would’ve put the software on the right directories without overwritting my existing software.

Still I was able to learn a lot of stuff. There is always something positive out of something bad.

1.-I was able to configure openldap from scratch, I installed a new cyrus-sasl from scratch, and berkeleydb 4 in order to get openldap started.
2.-I was able to configure an address book on my new ldap server. I was even able to query it using netscape and outlook express.
3.-Installing openldap was a pain on the ass but after hard work it worked and it was up and running.
4.- I hacked an init script to make openldap start.

Then after I realized that everything broke. I learned the following.

1.-Don’t use checkinstall and use make install only for certain packages specially if it will replace existing software. I had known this before but I was working on ldap for almost 48 hours and I was tired. Still don’t make the same mistakes again.
2.- I learned how to fix certain error messages using symlinks to libraries requested by programs that can’t find certain libraries.
3.-I learned to fix a broken linux server by removing rpm’s rebuilding rpms just so they can be removed.

here is a good tip on how to remove duplicate rpms

rpm bad-package -e –allmatches

4.-I was able to figure out what had caused all the damage to my existing programs and servers.

I had to uninstall openldap and the new cyrus-sasl software just to get my server working and although my ldap server is gone.

I was able to fix everything my server and everything is back to normal.

At least I know that I can build openldap again and without causing the sames problems.

:)

penguincares.


Apr 12 2007

Computer Hangs with using IE for Windows Updates and svchost.exe CPU 100%

Tag:tepezcuintle @ 14:28

I had the same kind of problems and these instructions solved it:

Either

1) turn off automatic updates, reboot, then manually go to windows
updates and install the updates, turn on automatic updates, reboot.

or (as in my case)

2) Perform these steps:

1. Click Start->Run, type “services.msc” (without quotation marks) in the
open box and click OK.
2. Double click the service “Automatic Updates”.
3. Click on the Log On tab, please ensure the option “Local System account”
is selected and the option “Allow service to interact with desktop” is
unchecked.

4. Check if this service has been enabled on the listed Hardware Profile. If
not, please click the Enable button to enable it.
5. Click on the tab “General “; make sure the “Startup Type” is “Automatic”.
Then please click the button “Start” under “Service Status” to start the
service.
6. Repeat the above steps with the other service: Background Intelligent
Transfer Service (BITS)

Step 4: Re-register Windows Update components and Clear the corrupted
Windows Update temp folder

1. Click on Start and then click Run,
2. In the open field type “REGSVR32 WUAPI.DLL” (without quotation marks) and
press Enter.
3. When you receive the “DllRegisterServer in WUAPI.DLL succeeded” message,
click OK.
4. Please repeat these steps for each of the following commands:

REGSVR32 WUAUENG.DLL
REGSVR32 WUAUENG1.DLL
REGSVR32 ATL.DLL
REGSVR32 WUCLTUI.DLL
REGSVR32 WUPS.DLL
REGSVR32 WUPS2.DLL
REGSVR32 WUWEB.DLL

After the above steps are finished. Sicne temporary folder of Windows Update
may be corrupted. We can refer to the following steps to rename this folder
that

1. Click Start, Run, type: cmd and press Enter. Please run the following
command in the opened window.

net stop WuAuServ
(note, you might need to reboot before the net stop command will work)

2. Click Start, Run, type: %windir% and press Enter.
3. In the opened folder, rename the folder SoftwareDistribution to SDold.
4. Click Start, Run, type: cmd and press Enter. Please run the following
command in the opened window.

net start WuAuServ

Hope this helps,


Apr 12 2007

Installing and configuring OpenLDAP for RedHat Enterprise Linux3

Tag:tepezcuintle @ 3:00

Original Link

(SimpleBind + SSL/TLS/start_tls + without-sasl + automount + netgroup + sudo + apache)

(See also related documents at http://web.singnet.com.sg/~garyttt/)

Credits: OpenLDAP is an OpenSource product brought to us by the OpenLDAP coreteam.

http://www.openldap.org/project/

Last Updated: 15-Sep-2006

Purpose:

This document describes the steps involved in installing and configuring an OpenLDAP Directory Server on RedHat EL3. This is to be accessed by RedHat or Solaris8/9 LDAP Client. Many useful productivity UNIX Shell scripts are also provided in this document.

Continua”Installing and configuring OpenLDAP for RedHat Enterprise Linux3″


Apr 11 2007

The Basics of IP subnetting

Tag:tepezcuintle @ 21:40

To understand how subnetting works, you first must have a good basic understanding of IP addresses. An IP
address is set of binary octets broken into quads. That definition may not have made any sense to you, so we
will try it another way. An IP address follows what is called dotted octet notation. There are four sets of
numbers in an IP address, and dots are used to separate them, for example, 192.168.0.1.

Let’s go a bit further into the binary octet discussion. Binary numbers are broken down by what is known
as positional notation, which works from right to left in increasing place values. Here is an example:

Continua”The Basics of IP subnetting”


Apr 11 2007

What is the best IP address range to use Internally?

Tag:tepezcuintle @ 21:37

How Network Address Translation saved the Internet

Which range of IP addresses should I choose for my LAN?

What to do if your LAN is larger or more complicated than normal

Detailed recommendations of how to assign the internal IP
addresses on a typical LAN

Tips on configuring a DHCP server on your network

How many static IP addresses should I get from my ISP?

The Prisoner on the Internet

 

How Network Address Translation saved the Internet

Continua”What is the best IP address range to use Internally?”


Apr 05 2007

Sendmail flags for troubleshooting + PHP mail()

Tag:tepezcuintle @ 21:33

Cool interactive sendmail troubleshooting.

if you want to view all the behind the curtains work done by sendmail
while trying to send an email you can try the following.

sendmail -v emailaddress@domain.com < file_you_wanna_send

here is am sending an email to imrobotmaker"@"yahoo.com

Continua”Sendmail flags for troubleshooting + PHP mail()”


Next Page »