Libvirt release 0.9.11 has added support for Open vSwitch so it is no
longer required to use bridge compatibility mode! Fedora 17 has 0.9.11
included.
If you have brcompat enabled disable it.
Configuration basically consists of adding the following line to a
interface setup in bridge mode with the bridge being a OVS bridge.
<virtualport type='openvswitch'/>
Here are a few links to point you in the right direction.
Configuring a domain's network interface to use Open vSwitch
http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Libvirt;hb=HEAD
Additional information on Open vSwitch Hypervisor integration.
http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=IntegrationGuide;hb=HEAD
If you have a version of libvirt older than 0.9.11 you will still need to use bridge compatibility mode as described in the previous post http://binarybitme.blogspot.com/2011/09/using-open-vswitch-with-libvirt-and.html
Saturday, July 21, 2012
Arch Linux ARM Post-Install Notes
Reinstalled Arch Linux ARM on my Pogoplug V3 today. Thought I would write down some notes for the next time I do it. Hopefully others find them useful. Just basic general stuff that is easy to forget. I may add to this over time.
After following http://archlinuxarm.org/support/guides/system/first-steps below are additional steps I take.
ssh-keygen -q -f /etc/ssh/ssh_host_key -N '' -t rsa1
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
adduser USERNAME
gpasswd -a USERNAME wheel
pacman -S sudo
visudo
Uncomment the line.
%wheel ALL=(ALL) ALL
Then save the file.
Then disable root login over SSH.
vi /etc/ssh/sshd_config
Change
#PermitRootLogin yes
to
PermitRootLogin no
For additional security I also recommend replacing password authentication with key based authentication. You can find more info on the Arch Linux Wiki (https://wiki.archlinux.org/index.php/SSH_Keys).
At this point you can either reboot or restart sshd and logout and back in.
You may run across times where you need more entropy on a system for pacman-key --init and other tasks haveged can generate it for you.
pacman -S haveged
haveged -w 1024
RUN TASK
pkill haveged
If no longer needed you can remove it.
pacman -Rs haveged
After following http://archlinuxarm.org/support/guides/system/first-steps below are additional steps I take.
General Arch Linux ARM Tips
Never perform a system upgrade (pacman -Syu) without out reading any announcements since your last upgrade otherwise there is a good chance you will break your install if you don't. These can be found for Arch in general at http://www.archlinux.org/, http://archlinuxarm.org/, and the forum for your specific device on http://archlinuxarm.org/forum/.Additional SSH Server Security
Generate New SSH Host Keys
Arch Linux ARM ships with SSH host keys already generated you need to replace these instead of using the ones shipped with Arch Linux ARM. Most distributions with generate unique keys automatically Arch Linux ARM does not do this for you. If you have issues with entropy on your device see entropy section below.ssh-keygen -q -f /etc/ssh/ssh_host_key -N '' -t rsa1
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
Disable Root Login Over SSH
Make sure you have another user on the system that can su/sudo.adduser USERNAME
gpasswd -a USERNAME wheel
pacman -S sudo
visudo
Uncomment the line.
%wheel ALL=(ALL) ALL
Then save the file.
Then disable root login over SSH.
vi /etc/ssh/sshd_config
Change
#PermitRootLogin yes
to
PermitRootLogin no
For additional security I also recommend replacing password authentication with key based authentication. You can find more info on the Arch Linux Wiki (https://wiki.archlinux.org/index.php/SSH_Keys).
At this point you can either reboot or restart sshd and logout and back in.
Generating Entropy
https://wiki.archlinux.org/index.php/Pacman-key#Initializing_the_keyringYou may run across times where you need more entropy on a system for pacman-key --init and other tasks haveged can generate it for you.
pacman -S haveged
haveged -w 1024
RUN TASK
pkill haveged
If no longer needed you can remove it.
pacman -Rs haveged
Tuesday, September 20, 2011
Using Open vSwitch with libvirt and QEMU/KVM
[UPDATE 7/21/2012]
I have written an updated blog post on Libvirt/Open vSwitch intergration now that Libvirt supports Open vSwitch directly since 0.9.11. If your libvirt isn't version 0.9.11 or higher you will still need to use bridge compatibility mode as described below. Otherwise check out the new post http://binarybitme.blogspot.com/2012/07/libvirt-0911-has-support-for-open.html
----
This is a quick and dirty write-up of using Open vSwitch(OVS) with libvirt and QEMU/KVM. I assume you already have libvirt(http://libvirt.org) up and running with QEMU/KVM. Other hypervisors may work but I haven't tried them at this time.
Grab the master of Open vSwitch(http://openvswitch.org/) with git because in previous versions there is an issue with removing ports from a fake bridge when the TAP is just deleted without first removing it from the bridge. This is what happens with libvirt and KVM when a VM is shutdown. This is fixed in the OVS master. The fix will also be in versions 1.2.2, 1.3.0, and later when released.
1) Compile and install OVS by following INSTALL.Linux
2) Compile and install OVS bridge compatibility mode by following INSTALL.bridge
Now you should have a working OVS install with bridge compatibility mode.
3) Create a fake bridge for each VLAN you want available. So say you have br0 as your real bridge and you want a VLAN 101.
ovs-vsctl add-br br101 br0 101
This will create a fake bridge tagged with VLAN 101. Do this for every VLAN you want.
4) In libvirt use a bridge interface setting the source to the appropriate fake bridge.
For example if you want a virtual machine on VLAN 101 assign it to br101
You can add the network interface by editing domain's XML directly with virsh.
<interface type='bridge'>
<source bridge='br101'/>
</interface>
Add this in the <devices> section. Libvirt will fill in the <mac/> and <address/>.
You can also add the network interface with virt-manager.
Now when you boot the VM libvirt will create the TAP interface and add it to br101. Since bridge compatibility mode is running OVS will add the port tagged with the appropriate VLAN when libvirt adds the interface to the bridge.
After starting the VM you should see the associated TAP interface when you list the ports in OVS.
ovs-vsctl list port
It should have the appropriate tag. In this case the tag should be 101.
Notice the port will not show up under br0 if you list-ports. You will have to list-ports on the fake bridge
ovs-vsctl list-ports br101
When you shutdown the VM the TAP interface will be destroyed and the port will be removed from OVS. If the TAP interface is destroyed, but the port still exists in OVS then you are not using a new enough version of OVS.
This will not help with mirrors or other advanced features but will give you the basic ability to automatically add and remove ports on the virtual switch with the appropriate VLAN tags for your virtual machines' interfaces.
I am working on a helper service that will enable the ability to automatically handle mirrors and other advanced functionality. It will also allow you to avoid using bridge compatibility mode if you desire. Eventually I hope to see OVS support built into libvirt but this will help until then.
I have written an updated blog post on Libvirt/Open vSwitch intergration now that Libvirt supports Open vSwitch directly since 0.9.11. If your libvirt isn't version 0.9.11 or higher you will still need to use bridge compatibility mode as described below. Otherwise check out the new post http://binarybitme.blogspot.com/2012/07/libvirt-0911-has-support-for-open.html
----
This is a quick and dirty write-up of using Open vSwitch(OVS) with libvirt and QEMU/KVM. I assume you already have libvirt(http://libvirt.org) up and running with QEMU/KVM. Other hypervisors may work but I haven't tried them at this time.
Grab the master of Open vSwitch(http://openvswitch.org/) with git because in previous versions there is an issue with removing ports from a fake bridge when the TAP is just deleted without first removing it from the bridge. This is what happens with libvirt and KVM when a VM is shutdown. This is fixed in the OVS master. The fix will also be in versions 1.2.2, 1.3.0, and later when released.
1) Compile and install OVS by following INSTALL.Linux
2) Compile and install OVS bridge compatibility mode by following INSTALL.bridge
Now you should have a working OVS install with bridge compatibility mode.
3) Create a fake bridge for each VLAN you want available. So say you have br0 as your real bridge and you want a VLAN 101.
ovs-vsctl add-br br101 br0 101
This will create a fake bridge tagged with VLAN 101. Do this for every VLAN you want.
4) In libvirt use a bridge interface setting the source to the appropriate fake bridge.
For example if you want a virtual machine on VLAN 101 assign it to br101
You can add the network interface by editing domain's XML directly with virsh.
<interface type='bridge'>
<source bridge='br101'/>
</interface>
Add this in the <devices> section. Libvirt will fill in the <mac/> and <address/>.
You can also add the network interface with virt-manager.
Now when you boot the VM libvirt will create the TAP interface and add it to br101. Since bridge compatibility mode is running OVS will add the port tagged with the appropriate VLAN when libvirt adds the interface to the bridge.
After starting the VM you should see the associated TAP interface when you list the ports in OVS.
ovs-vsctl list port
It should have the appropriate tag. In this case the tag should be 101.
Notice the port will not show up under br0 if you list-ports. You will have to list-ports on the fake bridge
ovs-vsctl list-ports br101
When you shutdown the VM the TAP interface will be destroyed and the port will be removed from OVS. If the TAP interface is destroyed, but the port still exists in OVS then you are not using a new enough version of OVS.
This will not help with mirrors or other advanced features but will give you the basic ability to automatically add and remove ports on the virtual switch with the appropriate VLAN tags for your virtual machines' interfaces.
I am working on a helper service that will enable the ability to automatically handle mirrors and other advanced functionality. It will also allow you to avoid using bridge compatibility mode if you desire. Eventually I hope to see OVS support built into libvirt but this will help until then.
Saturday, July 16, 2011
pfSense in a Virtualized Environment
Just a quick note on pfSense in a virtualized environment. For whatever reason pfSense hangs during installation in a VM with default VM settings on all hypervisors I have tried. I have seen this on 1.2.3 and 2.0-RC1 on VirtualBox, KVM, and Xen. I have not tested with pfSense 2.0-RC3. The way I found to get around this is to put the cdrom and the harddisk on separate virtual disk controllers. For example sticking one on a SCSI and the other IDE seems to work around the issue. I have tested this in both VBox and KVM.
If you haven't checked out pfSense it complimants Open VSwitch quite well in a virtualized lab environment as a LAN router and firewall. My favorite firewall/router distribution I have used hands down.
If you haven't checked out pfSense it complimants Open VSwitch quite well in a virtualized lab environment as a LAN router and firewall. My favorite firewall/router distribution I have used hands down.
VirtualBox 4 Networking Notes
I recently setup a lab environment on older hardware that didn't have VT-x. For this setup I used VirtualBox 4 headless, phpVirtualBox, and Open VSwitch. I have a few notes worth documenting in one place.
Not entirely related but I would love to see enhanced protection such as sVirt like functionally developed for libvirt/KVM on a system with SELinux or Apparmor available. That would go a long why to help protect the host and other guests if an escape from a guest was achieved.
- If you are trunking to a VBox VM use the vnic type of AMD PCNet PCI II (Am79C970A). This is not the default vnic type. I have been able to trunk to Linux and FreeBSD guests including pfSense using the PCNet II vnic with a trunk from Open VSwitch with no issues. Every other vnic type I tried didn't work for me. I haven't tried the virtio drivers on VBox they may work as well. I do know the virtio drivers work when trunking to a Linux VM on KVM.
- If you are setting the destination of a mirror in Open VSwitch to a VM vnic on VBox you will need to set extra data on the vnic to allow promisc mode to be enabled.
VBoxManage setextradata VM_NAME \
"VBoxInternal/Devices/pcnet/0/LUN#0/Config/IfPolicyPromisc" "allow-all"
Replace pcnet with the appropriate vnic type you are using. - VBox comes with VBoxTunctl so you don't need to install tunctl seperatly to create tap interfaces for use with Open VSwitch.
As a side note you don't get 64 bit guests or hardware breakpoints while debugging in a guest without VT-x on VirtualBox :-( but at least you get full virtualization with good performance :-).
Not entirely related but I would love to see enhanced protection such as sVirt like functionally developed for libvirt/KVM on a system with SELinux or Apparmor available. That would go a long why to help protect the host and other guests if an escape from a guest was achieved.
Subscribe to:
Posts (Atom)