Search This Blog

Thursday, November 9, 2017

Code Snippet on your Blog



Hello :)

While writing my previous post i was unaware how to copy code snippets (as their alignment have to be correct).

So here is how you do it

Go to HTML view of your blog and then add this at the header or at the top

<script language="javascript" src="https://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js" type="text/javascript">
<script language='javascript' type='text/javascript' src='https://google-code-prettify.googlecode.com/svn/trunk/src/lang-css.js' />

<script type='text/javascript'>
function addLoadEvent(func) {
  var oldonload = window.onload;
  if (typeof window.onload != 'function') {
    window.onload = func;
  } else {
    window.onload = function() {
      if (oldonload) {
        oldonload();
      }
      func();
    }
  }
}
addLoadEvent(prettyPrint);
</script>


and then paste your code snippet between the below mentioned tag

<pre class="prettyprint">
---your code here---
</pre>

Wednesday, November 8, 2017

VMware keystroke based automation (The Pythonic Way)




Hi all,

This is based on my another day at work.

I was stuck with an issue "Press any key to boot from CDrom" while automating the unattend windows installation using autounattend.xml

Ofcourse i could have cracked the ISO to fix this up but my company would n't agree to it :)
This really blew me down until i found the following link (URL)

https://www.virtuallyghetto.com/2017/09/automating-vm-keystrokes-using-the-vsphere-api-powercli.html

https://github.com/lamw/vghetto-scripts/blob/master/powershell/VMKeystrokes.ps1

By referring that, i was able to write my own python based implementation of the same (Only takes return"Enter" keystroke as this is my need)
The code snippet is pasted below, feel free to reuse them (pressEnter.py)

import atexit
import argparse
import getpass
import sys
import textwrap
import time

from pyVim import connect
from pyVmomi import vim


def get_args():
    parser = argparse.ArgumentParser()

    # because -h is reserved for 'help' we use -s for service
    parser.add_argument('-s', '--host',
                        required=True,
                        action='store',
                        help='vSphere service to connect to')

    # because we want -p for password, we use -o for port
    parser.add_argument('-o', '--port',
                        type=int,
                        default=443,
                        action='store',
                        help='Port to connect on')

    parser.add_argument('-u', '--user',
                        required=True,
                        action='store',
                        help='User name to use when connecting to host')

    parser.add_argument('-p', '--password',
                        required=False,
                        action='store',
                        help='Password to use when connecting to host')
    parser.add_argument('-n', '--name',
                        required=True,
                        action='store',
                        help='Name of the virtual_machine to look for.')

    args = parser.parse_args()

    if not args.password:
        args.password = getpass.getpass(
            prompt='Enter password for host %s and user %s: ' %
                   (args.host, args.user))

    return args

args = get_args()
si = connect.SmartConnectNoSSL(host=args.host, user=args.user,
                            pwd=args.password, port=args.port)

# doing this means you don't need to remember to
# disconnect your script/objects
atexit.register(connect.Disconnect, si)

# search the whole inventory tree recursively... a brutish but
# effective tactic
vm = None
entity_stack = si.content.rootFolder.childEntity
while entity_stack:
    entity = entity_stack.pop()

    if entity.name == args.name:
        vm = entity
        del entity_stack[0:len(entity_stack)]
    elif hasattr(entity, 'childEntity'):
        entity_stack.extend(entity.childEntity)
    elif isinstance(entity, vim.Datacenter):
        entity_stack.append(entity.vmFolder)

if not isinstance(vm, vim.VirtualMachine):
    print "virtual machine %s not found" % args.name
    sys.exit(-1)

print "Found VirtualMachine: %s Name: %s" % (vm, vm.name)

tmp=vim.UsbScanCodeSpecKeyEvent()
h="0x28"
hidCodeHexToInt = int(h, 16)
hidCodeValue = (hidCodeHexToInt << 16) | 0007
tmp.usbHidCode = hidCodeValue
sp = vim.UsbScanCodeSpec()
sp.keyEvents = [tmp]
r = vm.PutUsbScanCodes(sp)
print r

Here is how you run it.

python pressEnter.py -s <vmware host> -u <user> -p <password> -n <target vm> -o 443

and ofcourse you need pyVmomi library pre-installed :) :P

Monday, September 4, 2017

Supervisord Tweaks

Hello all,
"Necessity is the mother of invention"

As always while i was working on, i came across some requirements with supervisord. It was quite interesting and useful. So i thought i will blog it on.

Tweak 1: Send emails from supervisord when something is broken


Step 1: Install sendmail. You need this to send emails

# yum install sendmail supervisor

Step 2: Configure sendmail

# vim /etc/mail/sendmail.mc

Modify from

dnl define(`SMART_HOST', `smtp.your.provider')dnl
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl



to

define(`SMART_HOST', `<smtp server>')dnl
dnl # DAEMON_OPTIONS(`Port=smtp,Addr=<smtp server>, Name=MTA')dnl


Step 3: Install in your native python or virtual environment

# pip install superlance

Step 4: Configure crashmail as a part of your supervisord configuration

Add the section below to your supervisord conf

[eventlistener:crashmail]
command=/path/to/your/app -a -m <To email id> -s "/usr/sbin/sendmail -t -i"
events=PROCESS_STATE
buffer_size=500


Step 5: Restart supervisord service

# systemctl restart supervisord.service

With the above changes, an email is triggered when ever a process(s)/program(s) mentioned in the supervisord configuration file crashes or moves into fatal state.


Tweak 2: Creating process groups


Step 1: Lets assume our original supervisord configuration is as mentioned below

[program:process_worker1]

directory=/usr/share/test/
command=/usr/bin/test1
process_name=%(process_num)s
autostart=true
autorestart=true
stdout_logfile=/var/log/test/process_worker1.log
stderr_logfile=/var/log/test/process_worker1_err.log
environment=PYTHONPATH="/usr/share/test/",xyz="/opt/config.ini"

[program:process_worker2]
directory=/usr/share/test/
command=/usr/bin/test2
process_name=%(process_num)s
autostart=true
autorestart=true
stdout_logfile=/var/log/test/process_worker2.log
stderr_logfile=/var/log/test/process_worker2_err.log
environment=PYTHONPATH="/usr/share/test/",xyz="/opt/config.ini"

Step 2: Add these changes mentioned below to create a group service

[group:test_groups]
programs=process_worker1,process_worker2


Step 3: Restart supervisord

# systemctl restart supervisord.service

Step 4: Now to stop/start/restart/status a group, do the following

# supervisorctl -c supervisor.conf restart test_groups:*

This configuration lets us group multiple programs/process together in supervisord


Tweak 3: Limited Retries for failed programs


Supervisord tries to keep all the process/programs alive by making constant retries. There could be number of use cases, where we may need to limit the retries.

The highlighted values in the section mentioned below would describe about the parameters that would help us limit the retries

[program:test_worker]
directory=/usr/share/test/
command=/usr/bin/test_worker
process_name=%(process_num)s
autostart=true
autorestart=unexpected
startsecs=10
startretries=3
stdout_logfile=/var/log/test/test_worker.log
stdout_logfile=/var/log/test/test_worker_err.log
environment=PYTHONPATH="/usr/share/test/",xyz="/opt/config.ini"


startsecs - It means the number of seconds the system has to wait to consider the process running

startretries - It means the number of retries the process can take. But this retry is applied only on process which is not in running state

So in the example above if the program crashes (in less than 10 sec), the maximum retries that can happen is 3

Thursday, July 20, 2017

One line Simple FTP Server

Hey Guys,

As always this is an accidental discovery while working on some automation project.
How to create a FTP server in one line of command.

Here we go,
I will assume that you already have Python on your system :)

    python -m pyftpdlib -p 21 -w -d /some/dest/directory -D -n <Your IP>

If you wish to see the console output later in a file and push the process to background then,

    python -m pyftpdlib -p 21 -w -d /some/dest/directory -D -n <Your IP> > ftp.log 2>&1 &

you can ofcourse change the port from port 21 to some other.

Looks simple isn't it ???? 

Monday, May 22, 2017

Apply SSL Certificate via HAproxy


Hi Again,


Just thought to share, how a SSL certificate is applied to HAproxy LB

First we need to generate “pem” file which includes private key  from “pfx”. It promts for password

# openssl pkcs12 -in dummy.pfx -out dummy.pem  -nodes

Note: pem file which includes private key is important and not just pem file without private key content

Copy this "pem" file to a specific location, say like

# cp dummy.pem /etc/haproxy/ssl/

Configure SSL cert to Haproxy  - In the front end section

frontend www-https
    mode http
    option forwardfor
    option http-server-close
    bind 0.0.0.0:443 ssl crt /etc/haproxy/ssl/dummy.pem
    reqadd X-Forwarded-Proto:\ https
    default_backend web-backend

and Restart haproxy service

# service haproxy restart

Ola!!!!   When you access your application you must see something like this on the address bar J



Thursday, March 9, 2017

Centos - How to undo a package Installation - Yum 


Hi Friends,

Yesterday someone in my server, accidently updated libvirt packages while trying to install something else.

Puff !!!
All my VMs started failing on reboot
Identified that verions of libvirt and qemu does nt work anymore.

Ona  quick research learnt this

"YUM HISTORY UNDO <NUM>"

It is a life saver

I initially tried "yum downgrade" - but it usually failed in downgrading dependency.

So let me explain you the magic.


Step 1: Find the history

# yum history
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
ID     | Login user               | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
     8 | root <root>              | 2011-10-03 14:40 | Install        |    1   
     7 | root <root>              | 2011-09-21 04:24 | Install        |    1 ##
     6 | root <root>              | 2011-09-21 04:23 | Install        |    1 ##
     5 | root <root>              | 2011-09-16 13:35 | Install        |    1   
     4 | root <root>              | 2011-09-16 13:33 | Erase          |    1   
     3 | root <root>              | 2011-09-14 14:36 | Install        |    1   
     2 | root <root>              | 2011-09-12 15:48 | I, U           |   80   
     1 | System <unset>           | 2011-09-12 14:57 | Install        | 1025  

Step 2: Revert the change

# yum history undo 8

Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
Undoing transaction 8, from Mon Oct  3 14:40:01 2011
    Install screen-4.0.3-16.el6.i686
Resolving Dependencies
--> Running transaction check
---> Package screen.i686 0:4.0.3-16.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved
================================================================================
 Package          Arch       Version            Repository              Size
================================================================================
Removing:
 screen           i686       4.0.3-16.el6       @rhel-6-server-rpms     783 k

<snip>

Removed:
  screen.i686 0:4.0.3-16.el6
Complete!
Hope this helps someone. Ola ... have a great day

Wednesday, October 15, 2014

OpenStack - Do you know how to evacuate Instances from server which has gone down - "nova host-evacuate"

Hello again,

Recently my server running openstack-compute had to be brought down for maintenance (which was running some 60 VMs). This is when i learnt how to use "nova evacuate"

This is a cool feature in openstack using which you can get back instances from the server which has gone down. Ya, but it requires some prior configuration.

The important stuff is how you keep you root disk. Openstack allows three different configurations to fetch the root disk.

1. Root on volume (EBS volumes) - By this instance can be brought up in other server with the same EBS(root) volume.

2. Instances folder in shared mode using NFS, GlusterFS, etc - Basically the disk file needs to be accessed across all nodes, i.e mount point for instances folder must be shared across all nodes. GlusterFS appears to work super cool in this method.

3. Disks are cleaned but instances are recreated from fresh disk from glance - Even though the disk is ultra new, instance information (like IP, instance Id,etc.,) are retained just as it was before evacuation.

Once the server is down, all you need to do is issue the following command

# nova host-evacuate --target_host <server where the VMs need to be transferred> --on-shared-storage <server which got down>

Eg:

# nova host-evacuate --target_host old_node --on-shared-storage new_node

I have added a video demonstration of the entire flow. I hope this helps someone. Have a great day


                                                  :) :) :) :) :) :) :) :)