Empire Maintenance - COMPLETE

Discussion in 'Empire News' started by Aikar, Nov 17, 2014.

  1. Wait so are you saying that there will be 3 SEPERATE servers that hold 3 smps? Or are you saying u will be adding 3 smps?
  2. No smps will be added. But the actual servers that run those actual smps will be different.
  3. Wait so its still all 1 server right?
  4. They went and took this picture and sent it to me.... Yes still with same host, I really like them :)



    To the users like yourself, nothing is changing.

    Right now we have 5 servers purely to host the game servers, and were lowering it down to 3 (but not going to cause lag because they are better servers)

    Maint is in roughly 1 hour. I'm running a script every 5 mins that syncs the data from old to new server, so itll be very little to copy once the servers shutdown, to keep the downtime as minimal as possible.

    https://gist.github.com/aikar/6cc08e9a880fcac5f569
  5. You will notice no difference in gameplay.
  6. Awesome, but when will the server be updated to 1.8?
  7. Ok thnxs aikar ;) :)
  8. Start up the new hardware, transfer the final transactions. What could possibly go wrong?
  9. Now that is why they are the coolest host around :p
    Kephras likes this.
  10. There is too much that needs to occur within the minecraft multiplayer development community before we can update to 1.8 for us to give even a vague answer.
    Devartete and cadgamer101 like this.
  11. Getting close. I was reminded of another thing I've been wanting to do, to setup a big system configuration called Huge Pages.

    This WILL provide a massive performance boost, so I am finalizing setting that up to make sure the servers are running with the best possible speed.
    Olaf_C, sambish20 and Silken_thread like this.
  12. For some reason i have never experienced lag :)
    Even at the He's a Pirate song in the cow hockey arena.
    Maybe I'm just getting all the processing power.
  13. Yay! its back up!!
    (ps: first)
    also, it seems as if the default server has switched to smp2.
  14. the default server is constantly rotating every 5 minutes usually. You logged out on smp2 5 hours ago is why you logged back in there.
  15. Also yes, servers are now back up! Everything went as planned. No issues occured (other than me having to play around with Huge Page settings to get it working)

    EMC is now even more redundant, with Bungee running over 4 servers now randomly being used, so bungee going down can no longer take us down completely :)

    ==== Summary of changes! =====
    1) Moved from 5 Dedicated Servers powering the Game Servers to 3.
    - Newer servers are much more powerful and able to handle more load than the previous.
    - Now running 3 SMP's per server, with Utopia/Stage/Games residing on MasterDB
    - Still have 3 other servers: DB, WebProxy and Web
    2) Moved to a newer SSD drive.
    - Reset the timer for failure. SSD's fail a lot faster than HDD's, and MC and the like would reduce the life span even more from constant writes. Since those old drives were nearing 1.5 years old, they were getting to a risky age.
    - Newer drive with faster specs than previous
    3) Enabled "Huge Pages / Large Pages"
    - Lets the OS / Java access memory MUCH more effeciently, giving a substantial performance boost
    4) Spun up 3 more BungeeCord instances over the 3 new servers, and Round Robin them in.
    - Splits up load across multiple servers
    - Redundancy incase 1 crashes (and for some reason doesn't auto restart.
    - Resistance to network level issues on a single IP (multiple IP's)
    5) Database Server Upgrade
    - Updated to Percona 5.6, the latest stable with many changes to the system
    - Enabled some needed config changes to alleviate DB issues that we've seen some times.
    - Adjusted memory settings - DB was using too much which left less room for the other stuff, which made us have to leave stage offline for a while. (I was EXTREMELY generous before on memory not expecting to run other apps on the server, way more than was needed to load our entire DB into memory... which is unnecessary)

    Quite a lot of work... Time for me to sleep now.
  16. Good job Aikar!, sounds like a whole lot of work to be doing at 1am!
    You deserve the sleep!
  17. Nice to see that full 20 tps :D
    Nice work Aikar
  18. UPGRADE FTW
  19. There was a downtime in voting this morning, but I resolved it before I went to work. Forgot to open the votifier port on firewall.
  20. Awesome :D Love to see new hardware as always. Guessing this is running on Hivelocity's new spec?