Work has begun to migrate the DirectAdmin resellers server, gt20-1, to a virtualized environment with an anonymous hostname name. Customers will be emailed the hostname separately. The new v-server will brought up first, and DirectAdmin installed on it (this work has begun). Afterwards we will begin migrating accounts one by one to the new server. This does mean new your ip address will change. But as we migrate accounts we will be giving each account a private ip address just for them to compensate.

There will be a time that some accounts can access the Installatron feature. At first it will be accounts that have moved to the new server. But after we migrate Installatron’s license to the new ip addresses, it will be the accounts still on the old server… We will be happy to install any of those scripts for you or you customers. Just simply put in a ticket with the account details, and the script you would like installed.

We are currently lowering the TTL on all the dns entries, so that any caching of incorrect ip addresses will be minimal… We’ll reset it after the migration. During the individual migrations the existing server will show a message that the account is being migrated to the new server.

We have decided to virtualize all outward facing servers for the same reasons companies all over the world are doing so: administration benefits as well as better resource management. This will help us improve service, as well as keep out prices low.

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

At 9:40pm Central time, we were alerted to an intermittent packet loss problem by the nagios systems affecting the Dallas servers. Investigation revealed it was the last hop before the servers. A quick reboot of the switch appears to have recitified the problem. As of 10pm, we are not experiencing any packet loss from the monitoring systems. We will continue to monitor.

At 10:20, the problem started again. Checking across the switch in question shows the switch to be performing properly.  We have placed a Critical ticket with Virtbiz support to get them to look upstream from the switch.

10:30 – Virtbiz is looking into it but the loss disappeared again.  Problem came from one account that decided to start uploading porn… a violation of our TOS. The account has been terminated.

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

… to add the zip module at a customer’s request. Shouldn’t be long… ~ 1 hour. Most of the time the server will be available… occassionally WHM will take apache offline a couple of times.

1:30 pm – Done

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

… while we resize the filesystem for the new server. It shouldn’t be offline for long, expected to be less than 30 minutes.

Step 1 – Stop container – done
Step 2 – Unmount filesystem – done
Step 3 – check fs – Done
Step 4 – Resize filesystem – done
Step 5 - Resize logical volume – done
Step 6 – Remount filesystem – done
Step 7 – Restart container – done
Step 8 – Relax — working on it…

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

To our VPS customers,

Since the troubles with HyperVM/LXAdmin, we’ve been busy looking at alternatives. By far the most promising is turning out to be FluidVM from a company called Binary Karma.  Many of the features are very similar, and some are signifigant improvements, such as the ability to use iSCSI/SAN storage as drives for virtual machines. This is something we’ve wanted to move to anyway. And while technically it was possible with HyperVM, it wasn’t as integrated into the environment.

Backup procedures will be changing. All the Compute Nodes need to be reformatted anyway, and LVM installed on a couple that it wasn’t before. This will enable us to do backups without interrupting the virtual machines.

Dallas servers are going to be first to migrate to the new environment. Existing virtual machines will be migrated to new servers with little downtime. Occassionally, we do see issues with quotas and cpanel licensing but in both cases the fixes so far have been quick and easy.

Atlanta will take a little longer and will require some downtime while the OS gets re-installed and vm’s restored.  Customers wishing to migrate to Dallas will be given the option to do so beforehand. Though ip address changes will have to happen to migrate.  A date has not been set for this to occur…

All VPS customers hypervm/fluidvm, will receive new instructions with login details as they’re migrated.

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

We’re doing a live migrate on cpwhm1… this isn’t changing platforms or anything else just moving nodes. This is to clear off a node to begin the process of moving to the new platform.  We need a couple of clean nodes to work with on testing, but we’re excited to find and test out Binary Karma’s FluidVM product. In some ways their product is better aligned with where we want to go anyway… with things like iSCSI storage (hint: think fault tolerance).

The VM should only be unavailable for a few minutes during the process.

The live migrate went smoothly… cpwhm1 virtual server was offline from 6:19p to 6:24p. Re-applied cpanel license 1 minute later… All looks well…

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

You may have heard, the future of our provider of our virtualization management software, LXLabs, is in serious doubt at this point. The founder and developer has taken his own life, and there are several known vulernabilities in the software that have been made public recently. Some were fixed others remain in question. In response, we are have taken down the HyperVM service as well as LXAdmin/Kloxo on all our company vps’es. For customers running LXAdmin we strongly suggest doing the same.
 
Here’s how:
(log in with ssh as root to your vps)
service lxadmin stop   (stops the panel from working temporarily)
chkconfig –level 2 lxadmin off 
chkconfig –level 3 lxadmin off 
chkconfig –level 4 lxadmin off  
chkconfig –level 5 lxadmin off   (these turn off the service for those runlevels)
chkconfig –list lxadmin  (shows which level are ‘on’ … should all be off now)
(log out)
 
If there is something you need the panel for, we strongly recommend not leaving it running. Turn it on (service lxadmin start), do what you need, and then turn it back off (service lxadmin stop).
 
However this is not a long term solution. We are looking at the available virtualization solutions, and trying to work in which would provide a stable environment (something LXLabs’ needed improving on anyway), painless migration (as much as possible), and growth potential. We are heavily leaning to one of the more mature open source products. As this potentially gives us peer review of code asn actual access to the code itself. I am personally a 25 year veteran of software development, so I just feel better when I can get my hands on the code. Of course, we’ll do everything possible to ease migration pains when the time comes. But as we’re all about open communications with customers, we wanted to let everyone know the direction we’re heading and get your input if you have experience good or bad with a virtualization management platform. We’d love to hear your thoughts on the matter.
 
Customers wanting to come off the LXAdmin control panel to something else, please contact us. We can offer several alternatives – cp/whm, directadmin, dtc, ispconfig, etc. The only one we will charge for is cp/whm … namely because we can’t buy that product only lease it at $15/mo.  We will provide a cp/whm license to you at our cost if that’s the panel you desire. For DirectAdmin we will buy an internal license (ie stays with us if you leave) for your use for free as long as you’re with us. Some panels do not like being placed on an existing configured server. For those installs we’ll be happy to provide a second vps configured with the new panel free for a month in order to move accounts over to the new environment. Please feel free contact us to discuss your situation and how we can help.
 
Robert
Lagniappe Internet LLC

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

cPanel/WHM is not liking the kernel change after the move. It was running however when it went to update cPanel/WHM errors started showing up.  We’re restoring the previous environment, and will be moving the data over shortly. During the data move the virtual server including website will not be available (estimated @ 15-30 minutes).

The new plan to migrate to the new server will be to bring a new virtual server up, and then migrate accounts over to the new server one at a time.

22:56 – Data migration is completed. CP/WHM being reconfigured… so it may be flap up/down for the next 30 – 60 minutes. Will post when done.
23:21 — CP/WHM is back up.

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

Later on this evening we will be migrating the cpwhm1 virtual server to the GT24-2 node.  During that time, the virtual server will be down on 2 occassions. The first to take a final backup in preparation for the move and then again during the move. The first downtime should be short … less than 5 minutes based on previous backups. The other will be a little longer – between 30 and 60 minutes based on previous migrations.  All ip addresses and account specifics should move with the vps.

The new server features dual quad-core processors, 32GB of Registered ECC RAM, 4 Western Digitial RE2 hard drives connected to a LSI/Dell Perc 5si RAID controller with 128MB of dedicated RAM and battery unit… This is a signifigant upgrade for this vps. On top of the increased hardware, the new server utilized logical volume snapshotting for backups - meaning no nightly downtimes for backups (even though they were for only about 3 minutes… none is still better!)

This post will be updated as the migration progesses…

8:30 – vzDump begun
9:20 – 1st vps shutdown from vzDump 2nd sync begun
9:22 – vps back up – waiting for tar.gz to complete
9:50 – backup complete – validating file
10:15 – begining migration to gt24-2
11:17 – migration completed succesfully!!
11:30 — going to do a quick reboot to reconfigure memory allocations (increase it by 4x)
11:35 – Done…

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.

We just received this email from LXLabs:

 Dear Customers,

 Multiple security vulnerabilities were discovered in hyperVM and Lxadmin/Kloxo. It is recommended that you update your hyperVM/Kloxo systems to the latest version, as soon as possible.

 Details of the vulnerabilities will be posted in the coming days in our forum.

 On hyperVM or Kloxo master, Run:

 /script/upcp

 Lxlabs Support Team

We are updating the HyperVM system as I write this email… Please be sure to run your update. If you need help or would like us to do it for you, simply put in a ticket at http://www.lagniappeinternet.com/members/ and we would be happy to do so.

Lagniappe Internet

Robert

Robert Porter is the founder and managing member of Lagniappe Internet LLC. Robert holds multiple certifications incluging Oracle Certified Professional-Java 6 SE (OCP-J6), MCSE, A+, Net+, Project+, Security+, and multiple CIW certifications. He has been in the hosting industry for more than a decade and is founded Lagniappe Internet L.L.C. as a privately owned, completely debt free, hosting company based out of New Orleans. Robert's background includes 25+ years in programming, databases, networking and systems administration.