EveOnline - upgrading LMeve

As promised a few months ago, here's a guide on how to upgrade/update your LMeve installation to the lastest release. As usual, you will find the list of required below, and where to download them. Always use the lastest available. Those listed here were the ones available on the 8th of july.
This guide is by no way as complete as I would like, since I don't use a lots of functionality in LMeve, I may miss some points.
This is especially true for the LMeve database. I can't stress you enough, if your LMeve instance is used by multiple users and/or corporations, to backup your DB and/or snapshot your system prior any modifications.
You will find at the end of the article a way to re-inject your settings in the new LMeve DB.
If you are familiar with phpmyadm, most of the mysql stuff during this upgrade should be doable from that interface. I don't use it because I'm not familiar enough with mysql already, I don't want to add an other layer of fog in front of it :-)

If you have questions about LMeve per se, you should go to Lukas' dev site ((https://github.com/roxlukas/lmeve)
I'm just a lambda user of this tool, who struggled to install it the first time. These guides are there to help you out, but I'm definitively not an expert with LMeve.

Feel free to comment if you find these articles useful.



Enough talk, let's do it.

** files to download **
lmeve-0.1.57-beta.tar.gz    (https://github.com/roxlukas/lmeve/releases)   
YC-118-6_1.1_Icons.zip        (https://developers.eveonline.com/resource/resources)
YC-118-6_1.1_Renders.zip
YC-118-6_1.1_Types.zip
mysql-latest.tar.bz2        (https://www.fuzzwork.co.uk/dump/)



First, let's stop LMeve polling.
# crontab -l | sed 's/*/#*/' > /tmp/cron_temp; crontab /tmp/cron_temp; rm /tmp/cron_temp

***************************
*** Very important step ***
***************************
Backup LMeve BD. This is very important if you have an existing user database, tasks, messages that you want to keep in the new version.

# mysqldump -u root -p  --all-databases > /tmp/mysql_backup_20160708.sql
# gzip -9 /tmp/mysql_backup_20160708.sql

You may want to take a backup/snapshot of your VM now, before doing anything else.
# shutdown -h now
Take a snapshot/clone of your VM, and restart it.
In case something goes wrong for you, you will go back to that state afterwards.
(Taking a snapshot or cloning the VM is beyond the scope of this article)

Let's do the upgrade now.
As you will see, it will be way faster than a fresh install.

**************************************************************
* Naming convention for the rest of this article.
* My EVE DB is name "eve_aeg100_dbo", not the best name, but it was its name at the Aegis release.
* The LMeve specific DB is name, well, "lmeve" ;-)
**************************************************************


If you have followed my previous guide, your /opt/lmeve is a symlink to your current release of LMeve, thus, you could keep your current LMeve install in a separate directory while preparing the new one.

drwxrwxr-x  11 root root  1024 May 10 13:32 lmeve -> /opt/lmeve-0.1.54-beta
drwxrwxr-x  10 root root  4096 Jul 24  2015 lmeve-0.1.53-beta
drwxrwxr-x  10 root root  4096 Aug 24  2015 lmeve-0.1.54-beta

Stop mysqld and apache.    Mysqld isn't mandatory if you have only LMeve running on your VM. As I have multiple tools connecting to those DBs, I'd rather have them fail than fetching boggus data.
# systemctl stop mysqld
# systemctl stop httpd

# cd /opt
# cp /opt/lmeve/config/config.php /tmp/config.php
# rm lmeve
# tar xzvf ~user/lmeve-0.1.57-beta.tar.gz
# ln -s /opt/lmeve-0.1.57-beta  /opt/lmeve
# cp /tmp/config.php /opt/lmeve/config/config.php

To avoid too much garbage in the poller, edit your config.php to add the following lines:
$API_BASEURL="https://api.eveonline.com";
$CREST_BASEURL="https://crest-tq.eveonline.com";

(for example, at the end of the config file, just before the line "?>")


I do the following commands because my VM is running on my laptop which didn't have a full time Internet access, thus, I have a lot of garbage/errors which get logged for nothing, and which fill up my FS.
# cd /opt/lmeve/var
# rm -f http_errors.txt; ln -s /dev/null http_errors.txt
# rm -f poller.txt; ln -s /dev/null poller.txt

Extract images from CCP.
# cd /opt/lmeve/wwwroot/ccp_icons/
# unzip ~user/lmeve_upgrade/YC-118-6_1.1_Icons.zip
# mv Icons/* .
# ln -s items/* .
# rmdir Icons
# cd /opt/lmeve/wwwroot/ccp_img
# unzip ~user/lmeve_upgrade/YC-118-6_1.1_Types.zip
# mv Types/* .
# rmdir Types
# cd /opt/lmeve/wwwroot
# unzip ~user/lmeve_upgrade/YC-118-6_1.1_Renders.zip

LMeve application has been updated, now, let's take care of its DB, then, EveOnline DB.
*****************************************************************************
* WARNING ** WARNING ** WARNING ** WARNING ** WARNING ** WARNING ** WARNING *
*****************************************************************************
** As I was writing these lines, I realised that it is not the optimum way to do it.
** In my procedure, I drop the whole LMeve DB, which mean, destroying all users, roles, messages, tasks, etc.
** Since I don't use them, I have never pay attention to that issue.
** At the end of this guide, you will find a way to re-import the content these tables from your backup into your new LMeve DB.
*****************************************************************************

# systemctl start mysqld
# cd /opt/lmeve/data
# mysql -u root -p
mysql> drop database lmeve;
Query OK, 58 rows affected (0.35 sec)

mysql> create database lmeve;
Query OK, 1 row affected (0.00 sec)

mysql> quit
Bye
# mysql -u root -p lmeve < schema.sql
Enter password:
#

There's three way to import your corporate key into LMeve:
_ the GUI way, this function was added in LMeve-0.1.55 : See http://pozniak.pl/wp/?p=9577 for a picture (at the end of his article)
_ via phpmyadmin (http://your.lmeve.ip/phpmyadmin. Sorry, I never used this tool, I can't help you with that. But I've been told that it's quite intuitive.
_ via the command line, which is, of course, my prefered way :-)
# mysql -u root -p lmeve
mysql> insert into cfgapikeys values (1,'1234567','api-key-here');
Query OK, 1 row affected (0.01 sec)



# cd ~user/lmeve_upgrade
# bunzip2 -c mysql-latest.tar.bz2 | tar xf -
# cd sde-20160704-TRANQUILITY/
# mysql -u root -p eve_aeg100_dbo < sde-20160704-TRANQUILITY.sql

If you don't need to re-import your previous user DB, you could still re-create the admin/admin login/passwd for LMeve:
# php /opt/lmeve/bin/passwd.php

The last steps, is to remove the INSTALL file in /opt/lmeve/ and restore the crontab.
# rm /opt/lmeve/INSTALL
# crontab -e
 remove the # in front of the poller, save and exit.


That's it, we just have to restart the httpd, LMeve has been upgraded to 0.1.57-beta.



*************** IMPORTING PREVIOUS LMeve data (users, roles, etc) *****************************

At writing this upgrade guide, I noticed that it couldn't be complete if I don't deal with the existing informations such as users, roles, tasks and so on. Since I don't use messages nor tasks, I will do only the user/passwd part. For messages and  tasks, it will be same procedure, but with differents tables.

To restore our user-base and roles, we will recover our DB dump we did at the begining of this procedure, and uncompress it.
# ls -la /tmp/mysql_backup_20160708.sql
-rw-r--r-- 1 root root 325553334 Jul  8 09:56 /tmp/mysql_backup_20160708.sql

First thing first, make a copy :-) better worry than sorry.
Then edit your copy, and search for "LOCK TABLES `lmuserroles`", copy paste the lines between "LOCK TABLES" and "UNLOCK TABLES" (included) and add them in a file (inject.sql) Do the same research for lmusers, lmchars, lmconfig and for every other tables you want to restore (messages, lmtasks)


You will have to change the lmuserroles entry from:
    INSERT INTO
to
    INSERT IGNORE INTO
Because at lmuserroles creation (via schema.sql), an initial entry is created. It may be the case for some other tables. You will have to check in the /opt/lmeve/data/schema.sql.



It will look something like than in the end:

# cat inject.sql
LOCK TABLES `lmuserroles` WRITE;
/*!40000 ALTER TABLE `lmuserroles` DISABLE KEYS */;
INSERT IGNORE INTO `lmuserroles` VALUES (1,1),(2,1);
/*!40000 ALTER TABLE `lmuserroles` ENABLE KEYS */;
UNLOCK TABLES;

LOCK TABLES `lmusers` WRITE;
/*!40000 ALTER TABLE `lmusers` DISABLE KEYS */;
INSERT INTO `lmusers` VALUES (1,'admin','9fdd8383c10e946350e81c9202f08eba','192.168.51.1','30.07.2015 19:27',0,'css/rixxjavix.css',1),(2,'user','ce1ad3325b7ac3dbc0f230fcf876feee','192.168.51.1','08.07.2016 11:35',6,'css/crius.css',1);
/*!40000 ALTER TABLE `lmusers` ENABLE KEYS */;
UNLOCK TABLES;

LOCK TABLES `lmchars` WRITE;
/*!40000 ALTER TABLE `lmchars` DISABLE KEYS */;
INSERT INTO `lmchars` VALUES (XXX,2),(YYY,2),(ZZZZ,2);
/*!40000 ALTER TABLE `lmchars` ENABLE KEYS */;
UNLOCK TABLES;

... etc etc


Inject these in your mysql DB:
# mysql -u root -p lmeve < inject.sql
# mysql -u root -p lmeve

mysql> select * from lmuserroles;
+--------+--------+
| userID | roleID |
+--------+--------+
|      1 |      1 |
|      2 |      1 |
+--------+--------+
2 rows in set (0.00 sec)

mysql> select * from lmusers;
+--------+-------+----------------------------------+--------------+------------------+-------------+-------------------+------+
| userID | login | pass                             | lastip       | last             | defaultPage | css               | act  |
+--------+-------+----------------------------------+--------------+------------------+-------------+-------------------+------+
|      1 | admin | 9fdd8383c10e946350e81c9202f08eba | 192.168.51.1 | 30.07.2015 19:27 |           0 | css/rixxjavix.css |    1 |
|      2 | user  | ce1ad3325b7ac3dbc0f230fcf876feee | 192.168.51.1 | 08.07.2016 11:35 |           6 | css/crius.css     |    1 |
+--------+-------+----------------------------------+--------------+------------------+-------------+-------------------+------+
2 rows in set (0.00 sec)

Once re-injected, your users should be able to connect as usual (I tried with my own user, it worked fine)

July update - Eve on backburners due to wardecs and summer.

Hi fellow pilots,

Just a quick report on my activities in those past 2 months. After a string of useless WarDecs by griefers, most of my activities were tunned down. I moved my BPO research activities to a couple of NPC stations for my main characters, and I trained two PI alts to have 6 research slots (most of their research skills are at IV and not V).
So, now, I have 40 research slots available, and as of today, 38 of them are in use. I kept those two ones free to move some 30+ days research to that alt.

As a side effect of World War Bee, it seems that some goons minded guys have moved from Null to HighSec, and are now having fun at ganking T1 hauler, and occasionally Freigher. They have their base of operations a few jumps away from Jita, and are pretty successful at that.
They have a non incorporated spy siting at a gate in a Gnosis (in a NPC corp), scanning Haulers and Freighters passing by. In the meantime, the gankers are waiting, in the next system down the pipe, inside a POS forcefield, a mere 500k km from the gate. As soon as a suitabe target is spoted, they align to the gate and start their count-down. Most T1 haulers have a 3 AU/s warpspeed, so it's easy to know when it will arrive on the gate, thus they warp precisely to arrive a few seconds before their victim jump in, and a couple of seconds before Police and Concord become active.

So, I'm now pretty reluctant to use my freighter to move stuff. I've more than 2B of minerals sitting in Jita, for less than 300 000m3, that I'm not willing to move myself. I will have to use some courrier contracts to move them past that "choke" point. The issue will be the same while I will try to move my production back to Jita.

That's the situation for the past 4 weeks. In the meantime, I took some vacactions with my wife, and simply filled my research slots with BPOs to research. As I went back I noticed that a lot of my Buy Orders and Sell Orders were gone, and I was left with a few billions siting in my wallet. Due to those pirates activities, I realized that I was loosing money by having that money just siting there.
I decided to go on a BPO shoping spree, and bough some for more than 4.5 Billions.
I have now less than 300m ISK left in my wallet, and barely playing Eve at all. I spend more time around the pool with a beer than on my computer.

I did tried the Serpentis event, but got bored after a couple of hours. IMHO, the design of this event is flawed, the event revolve around the principle of grinding. People with the most free time, will stand a chance, casuals like myself, don't.
But I must say that it was an ok way to make money, at around 3m ISK per site (the biggest one) and an additional 1m via salvage. I was using a drake with heavy missiles and a caracal with ligh missiles, I must say that clearing those sites was fast. For salvaing, I was using a catalyst with 4 salvagers and 4 tractors + a MTU. that was nicely efficient.

Now, for the next couple of weeks, here's the plan:
_ update my LMeve VM. The Citadel release updated a couple of stuff in CREST, and my old LMeve release seems unable to cope with these.
_ concoct a plan to keep building and selling stuff, without using my freighter. I will have to use courrier once again, so, I will have to adapt my calculations with these costs.
_ code a prevision tools, and an alert tool, to spot oportunities on the market. I'm collecting stats since january, and I have yet to build something to use them.
_ find an other system, in an other region, to start over. Jita is the biggest market for sure, but there's way too much goon-minded pirates in the surrounding regions.
_ Play the market. Since I'm now cash striped, I'm playing the market. Buying items at below my own building costs (sometimes waaay below), and waiting for the price to recover before reselling. I could take weeks, but I'm a very patient guy :-)
_ spend as much time as possible around the pool, with a good book or a few friends and beers, before the winter make its return.

To be honnest, the last goal is the one with which I've the more successes. :-)

Until next time,
Fly safe o7