Finding serial console log of a nova instance

The other day I was debugging why a volume booted nova instance doesn’t get dhcp leases. For that I need to find networking information of the instance during boot. Fortunately, nova attempts to log the boot details to a serial console log. To find where it’s located..

First, let’s list the instances which we care about:

$ nova list | grep f17
| 220d6e57-f7ad-4256-a861-7855ffc6788f | f17-builder       | ACTIVE  | novanetwork=192.168.32.4 |
| f1169841-5e67-45d9-85c4-0d44bd3bec90 | f17_volume_backed | ACTIVE | novanetwork=192.168.32.6 |

Then, grep that instance UUID in /etc/libvirt/qemu/* to find what’s the libvirt id of a specific nova instance:

$ grep f1169841-5e67-45d9-85c4-0d44bd3bec90 \
  /etc/libvirt/qemu/* 
/etc/libvirt/qemu/instance-00000007.xml:  f1169841-5e67-45d9-85c4-0d44bd3bec90
/etc/libvirt/qemu/instance-00000007.xml:      f1169841-5e67-45d9-85c4-0d44bd3bec90

$ virsh list | grep instance-00000007
 10    instance-00000007              running

From above, the libvirt instance is called instance-00000007

To find path to the serial console log of the instance, grep its libvirt xml information for the relevant fragment:

$ virsh dumpxml  instance-00000007 | grep -i console
      <source path='/var/lib/nova/instances/instance-00000007/console.log'/>
    <console type='file'>
      <source path='/var/lib/nova/instances/instance-00000007/console.log'/>
    </console>

The console log of the cirros image runs a bunch of useful debug commands after booting an instance — which gave me an insight that it wasn’t getting dhcp leases. Listing the commands it ran from the console.log for reference:

$ /etc/rc.d/init.d/sshd start
$ ifconfig -a
$ route -n
$ cat /etc/resolv.conf
# pinging nameservers
$ uname -a
$ dmesg | tail
$ tail -n 25 /var/log/messages

Also, for reference, relevant xml snippet of the console information:

$ virsh dumpxml instance-00000003 | \
  egrep -i 'serial type' -A9
    <serial type='file'>
      <source path='/var/lib/nova/instances/instance-00000003/console.log'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='1'/>
      <alias name='serial1'/>
    </serial>
    <console type='file'>
      <source path='/var/lib/nova/instances/instance-00000003/console.log'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>