The www.lagniappeinternet.com website has changed servers.
Please note if you are logging into DirectAdmin please use http://linux1.lagniappeinternet.com:2222/ or http://yourdomain.com:2222/ to log in! http://www.lagniappeinternet.com:2222/ will no longer work!
We’re in the process of moving the www.lagniappeinternet.com including the members support website (http://www.lagniappeinternet.com/members/) to a virtual server of it’s own and off the linux1 server.
This is designed to:
1) provide the customer with a better support experience as it won’t be tied to any customer servers.
2) the support site will be able to stay up during the migration of linux1 to gt20-1 server.
3)offload some of the processing overhead from linux1
During this move, the www.lagniappeinternet.com website and support site may be unavailable for short periods of time. Also the SSL certificate will have to be re-keyed and re-generated… so expect SSL warnings during that time.
ClientExec (the support website software) will also be upgraded to the release version of 3.2 during this move.
Update 3:10AM… Much of the work is done. The www.lagniappeinternet.com website has been moved including the customer support website. The ClientExec update has been done as well. Pointers were placed on the both machines for the services placed on the other to help keep people from getting lost.
A little work on the website remains… status2k needs to be moved, craftysyntax needs re-installing, etc. But generally all services are available.
Our provider, virtbiz, appears to have a connectivity issue. Will update as we get information back from them.
Update – an electrical problem led to the outage… Virtbiz has corrected the issue.
To help a customer out we’re suspending backups for VPS accounts for 1 day… We’re doing a whole vps restore for them, and don’t want to risk the backup removing files that don’t exist at the moment. VPS backups will resume tonight (Wednesday) at 10pm Central time.
We have been asked a couple of times about “automatic install scripts”. There’s really only one we’d consider – Installatron as it is one of the best at being kept current. We’re considering if this is a worthwhile option and would like to hear from you. With the new server coming online in the next couple of weeks, it’s made us reconsider offering this. So please let us know by comment here, putting in a ticket or just emailing me at rob -at- lagniappeinternet.com … Thanks!
All servers in Dallas are being moved over to the new PDU and will experience momentary outages while the power is re-routed. Each server should only be unavailable for a few minutes while the server is shutdown, power re-routed, and restarted.
Sorry for the inconvenience, however this will be a much better scenario for control of the servers in the long run.
Edit 6:30pm – All PDU moves are done… Sorry for the inconvenience, this is a positive move in the long run though. Thank you!
A replacement for Linux1 is coming soon. The current Linux1 server (a Supermicro 1U server with single Pentium4 2.4GHz, 2GB of RAM and two 300GB drives in RAID1) will be replaced with a Tyan GT20 Dual Xeon 3.0 processors, 4GB of RAM and 4 drives in RAID10) . While the load on the current Linux1 server is quite small (normally under 0.10), it was never our intention to keep this server long term. The new server had already been purchased and was waiting as a spare for the need to arise. Processors, RAM and drives will be arriving for it shortly. When they do a multi-week process of configuration and burn in will begin. It will then be sent to the Dallas d.c. for fitting into the server rack. Once there and powered on, it will remain there unused for about a week to verify nothing happened during shipping. Once all of that is done, we will begin the migration of accounts from one server to the other. We’ll attempt to do the migration during the lowest usage time of the day to minimize impact. Affected customers will be getting emails regarding account moves as we get closer to the anticipated migration date with more details. We simply wanted to put this early “warning” out there as soon as possible to give as much advanced notice as we could, and keep with our company philosophy of open, direct, and sincere communication with our customers.
The anticipated migration date is around April 15th.
If you have any questions, please do not hesitate to ask. Either place a support ticket at http://www.lagniappeinternet.com/members/ or email me at rob -at- lagniappeinternet.com
Quick Update: The drives have been picked and ordered… This system will use 4 Western Digital Caviar RE2 ”Enterprise class” 750GB 7200RPM, 16MB Cache SATA/300 drives in RAID-10 for speed and data integrity. “RE” on these drives standing for “RAID Edition”… These drives are optimised to run in RAID configurations on servers with a Mean Time Between Failure of 1.2 million hours. For a review of these drives check out http://www.pcworld.idg.com.au/review/servers_storage/western_digital/re2_wd7500ayys/220791
What WD says about these drives:
“Superior reliability – Designed and manufactured to server-class standards to provide best-in-class enterprise reliability in high duty cycle environments. With 1.2 million hours MTBF, these drives have the highest available reliability rating on a high-capacity drive. “
Sorry to borrow the title from the book series, but it seemed to fit.
We’ve been attempting to actually improve automatic monitoring and problem resolution and it has a negative impact… The other day we started swapping the Power Distribution Unit (PDU) out in Dallas to one that actively monitors and can restart servers if one fails to respond. The one server it was being tested on failed to respond and after attempting for 2 minutes the new PDU reset the server. On restart, the server decided the filesystem needed checking and mounted it in read-only mode, but the system came back up. The other monitoring systems check for web, smtp and pop email, as well as mysql connections being accepted. And within minutes of the reset, all of them were once again accepting connections. The problem was the file system was in read-only mode - so no data could be written. Because connections were being accepted, and web pages were being served (although many would have had errors if they required any writes or database access), the monitoring failed to accurately portray the true state of the server.
Connected events affected our email, voicemail (tied to email via a virtual pbx system) and the ticket/helpdesk system which integrates tightly with email.
Changes we will be making to address this:
- First, the lagniappeinternet.com website and support software will be moved to a virtual machine. The vm will be setup to be automatically replicated to the Atlanta D.C. on a periodic basis. With a DNS update this would bring the ticket/helpdesk back up very quickly.
- We’re investigating how to change monitoring so that it can detect a disk/filesystem fault such as read-only status. Some customers may already be aware we’ve been working on developing an ‘active’ or ‘smart’ monitoring system of our own… This experience will be taken into consideration on that product as it’s the scenario that other monitoring systems don’t catch.
- We’ve added third party routing for voicemails to support. This will let us get voicemail messages even if email is offline.
A PDU is a power distribution unit (a glorified power strip) while the one there was much nicer than what you would find in any hardware store, or even most computer stores, it still had few features other than monitoring total amperage draw and being rack mounted.
The new unit is an intelligent device and allows monitoring, remote rebooting, etc. It is being installed as I type this…
The servers themselves are running fine, however the upstream provider is working on a connectivity issue. We should hear word from them shortly.
Update:
Good morning,
The reason that you briefly lost connectivity earlier this morning is because the switch that connects your equipment to our network was rebooted by error. We found that the power feeds to the equipment were not as documented so when we removed power to one of the 2 supplies to undergo maintenance that actually resulted in a shutdown of the switch. We have fixed that issue from recurring. In all, it looks like you suffered disconnection of just about 1 minute.
I’m sorry for the issue. Please let me know if you have further questions.
Chris – Virtbiz
Comments Off