Any way to convert legacy (.txt) logs into something useful? - Printable Version
-Shoutbox (https://shoutbox.menthix.net)
+-- Forum: MsgHelp Archive (/forumdisplay.php?fid=58)
+--- Forum: Messenger Plus! for Live Messenger (/forumdisplay.php?fid=4)
+---- Forum: WLM Plus! General (/forumdisplay.php?fid=23)
+----- Thread: Any way to convert legacy (.txt) logs into something useful? (/showthread.php?tid=94801)
Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 12:42 PM
hi, I recently discovered a whole bunch of old logs created by messenger plus (between 2003 and 2005) all saved in plain text. I have been using Pidgin for a while, and I was wondering if anyone knew a fast way (automated) to convert these legacy logs to another format.
Looking at the logs, there is quite a bit of information missing, so I'm guessing a script will need to ask questions during conversion:
code:
.--------------------------------------------------------------------.
| Session Start: 26 February 2004 |
| Participants: |
| My user name (**********@hotmail.com) |
| Recipient's user name (*******@hotmail.com) |
.--------------------------------------------------------------------.
[16:44:33] My user name: Hello there
[16:45:02] Recipient's : hi
[16:45:49] My user name: How are you?
[16:46:37] Recipient's : I'm ok
[16:46:38] Recipient's : yourself?
So, I'm guessing the script would have to look at that and find the two participants, then ask which user starts with "Recipient's " as the user names get cut off!
I can't find a script or program that does this, and converting the logs by hand takes hours/days/weeks!
Anyone else tried attempting something like this?
Cheers
RE: Any way to convert legacy (.txt) logs into something useful? by matty on 06-18-2010 at 12:46 PM
To my knowledge no one has. There is just too much involved and too much that could potentially go wrong.
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 01:19 PM
quote:
Originally posted by matty
To my knowledge no one has. There is just too much involved and too much that could potentially go wrong.
I can't really see what could "go wrong" but I agree that it would be quite difficult to do. If I could program, I'd give it a shot as I know there's enough information in those logs to, at least, be able to convert to a different format.
RE: Any way to convert legacy (.txt) logs into something useful? by matty on 06-18-2010 at 01:24 PM
But in reality what is it that you are trying to achieve? Are you trying to mimic the new HTML logging format in Plus!? Are you trying to make the currently logs more colour rich HTML files?
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 01:42 PM
I'm trying to convert them to HTML, as used in Pidgin, so all my logs are in one place, and in the same format. Which makes them easier to search later on.
RE: Any way to convert legacy (.txt) logs into something useful? by Chrissy on 06-18-2010 at 02:00 PM
Change the file name from .txt to .html, then it's in HTML format 
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 02:05 PM
haha, I wish it was that simple! Pidgin's logs need to be formatted a specific way, or then it won't read them! Also, each conversation needs to be in a separate file.
RE: Any way to convert legacy (.txt) logs into something useful? by djdannyp on 06-18-2010 at 02:08 PM
I don't think you're going to be able to convert them to the exact same format as Pidgin to be able to read them in it properly. I think you're best off just accepting that it's not gonna happen and leaving the old logs in text format and just being able to get them either via Messenger Plus! Live's chat logger (if you still use it) or by just opening them in notepad
RE: Any way to convert legacy (.txt) logs into something useful? by Chrissy on 06-18-2010 at 02:08 PM
It can be done using various scripts but It's not worth the hassle as said above.
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 02:14 PM
quote:
Originally posted by Chrissy
It can be done using various scripts but It's not worth the hassle as said above.
Various scripts? Do you know where I can check these out?
@djdannyp: They don't need to be perfect, but there is actually enough information in them to recreate a readable log. I have done some manually, it just takes a really long time!
RE: Any way to convert legacy (.txt) logs into something useful? by Chrissy on 06-18-2010 at 02:17 PM
I just mean something you can code. When it's fun it changes various things (formatting it) and making it work.
If anyone had the knowledge and time to do it they could, but I don't think anyone will, sorry 
RE: RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-18-2010 at 02:36 PM
The formatting doesn't really bother me, as long as the text is all still there 
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 06-19-2010 at 04:32 PM
In theory, a script would run something like this:
code:
Search for the following string: "| Session Start:"
search for string for next occurrence of: ".--------------------------------------------------------------------."
if number of lines between these strings > 2 then ignore (as multi conversations will be much harder to convert)
else, detect your username from first line by searching for an email and store as $localuser
detect recipient username from second line by searching for an email and store as $recipientuser
convert date from "Session Start:" to the YYYY-MM-DD format and store as $sessiondate
search for first time stamp from the first message ( [HH:MMS] ) strip the brackets and store as $sessionstart
Print the following into an html file
<html><head><title>Conversation with $recepientuser at $sessiondate $sessionstart on $localuser (msn)</title></head><body><h3>Conversation with $recepientuser at $sessiondate $sessionstart on $localuser (msn)</h3>
Next, identify all unique screen names (aliases) from each line of the conversation and ask which username it refers to (as the text logs dont make it clear)
Then identify status messages ("user has changed their name to" or "user has gone offline") by looking for messages starting with a *
Format all local user messages with the following HTML:
<font color="#16569E"><font size="2">(TIMESTAMP)</font> <b>SCREEN NAME (TRUNCATED)/b></font> <font sml="MSN">CONVERSATION TEXT</font><br/>
Format all recipient messages with the following HTML:
<font color="#A82F2F"><font size="2">(TIMESTAMP)</font> <b>SCREEN NAME (TRUNCATED)
Format all status message with the following HTML:
<font size="2">(TIMESTAMP)</font><b> SCREEN NAME (TRUNCATED). STATUS.</b><br/>
close the file with:
</body></html>
then save the file with the following date format: $recipientuser/YYYY-MM-DD.HHMMSS.html
But I have no idea how to code it!
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 08-01-2010 at 04:27 PM
bump?
RE: Any way to convert legacy (.txt) logs into something useful? by Phil_123 on 08-05-2010 at 06:57 AM
PM me a legacy log and a Pidgin log, I can try to code one.
RE: RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 08-05-2010 at 11:46 AM
quote:
Originally posted by Phil_123
PM me a legacy log and a Pidgin log, I can try to code one.
Sent as PM
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 09-11-2010 at 05:42 PM
heard nothing from Phil_123, bumping again
RE: Any way to convert legacy (.txt) logs into something useful? by Chrissy on 09-12-2010 at 03:02 PM
Hey,
Obviously nobody is interested or they would of posted.
If you REALLY need it done offer some cash and then you'll get a lot of contenders that will raise their hand.
Good Luck
RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 10-03-2010 at 12:30 AM
I coded the most basic, rudimentary program in one hour while I was eating noodles. Then, I had to had to work on it some more. However, it is still not complete. Some parts need fixing, cleanup, and whatever.
I need more examples and more testing, for example, files with status messages in legacy format and HTML.
Is this what you want?
code:
<html><head><title>Conversation with Recipient's user name at 2004-02-26 16:44:33 on My user name (msn)</title></head><body><h3>Conversation with Recipient's user name at 2004-02-26 16:44:33 on My user name (msn)</h3><font color="#16569E"><font size="2">16:44:33</font> <b>My user name
</font><br/><font color="#A82F2F"><font size="2">16:45:02</font> <b>Recipient's
</span></strong></span></font><br/><font color="#16569E"><font size="2">16:45:49</font> <b>My user name
</font><br/><font color="#A82F2F"><font size="2">16:46:37</font> <b>Recipient's
</span></strong></span></font><br/><font color="#A82F2F"><font size="2">16:46:38</font> <b>Recipient's
The HTML doesn't validate.
Check the HTML file and the Java source code (...\src\) in the "Messenger Plus! Legacy Chatlog.zip" file.
It's my first public program, so maybe someone else can do better than me.
Also, you seem not to like that the names are getting cut off. Do you want them to be the full name and not get cut off?
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 10-03-2010 at 10:22 AM
Cheers, I'll check that out.
In the case of Pidgin, the HTML will not validate, but needs to be laid out the way it's expecting or it doesn't seem to read them at all!
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 10-03-2010 at 11:20 AM
Right, I've had a play with the script, there are a few bugs (for example, if an emoticon was used in a display name, the script can't find email addresses).
I have changed how the output is laid out, this makes Pidgin display the log correctly, but sadly, due to my lack of Java knowledge, I don't know how to make the file name show the date of the conversation, rather than the date of conversion.
Here is the patch that needs to be applied to the Java source XP1 attached:
code:
--- Original.Main.java 2010-10-02 20:10:14.000000000 +0100
+++ Main.java 2010-10-03 12:18:12.084575152 +0100
@@ -253,7 +253,7 @@
//File aFile = new File(fFileName);
- strOutput += "<html><head><title>Conversation with " + strArrayParticipants[1][0] + " at " + dfYearMonthDayTime.format(dateSessionStart) + " on " + strArrayParticipants[0][0] + " (msn)</title></head><body><h3>Conversation with " + strArrayParticipants[1][0] + " at " + dfYearMonthDayTime.format(dateSessionStart) + " on " + strArrayParticipants[0][0] + " (msn)</h3>";
+ strOutput += "<html><head><title>Conversation with " + strArrayParticipants[1][1] + " at " + dfYearMonthDayTime.format(dateSessionStart) + " on " + strArrayParticipants[0][1] + " (msn)</title></head><body><h3>Conversation with " + strArrayParticipants[1][1] + " at " + dfYearMonthDayTime.format(dateSessionStart) + " on " + strArrayParticipants[0][1] + " (msn)</h3>" + "\n";
if (alistContactTimestamp.size() == alistContactName.size() && alistContactTimestamp.size() == alistContactMessage.size()) // If contact timestamp, name, and message are all matching
{
@@ -261,11 +261,11 @@
{
if (strArrayParticipants[0][0].contains(alistContactName.get)) // local user
{
- strOutput += "<font color=\"#16569E\"><font size=\"2\">" + dfTimestamp.format(alistContactTimestamp.get
+ strOutput += "<font color=\"#16569E\"><font size=\"2\">(" + dfTimestamp.format(alistContactTimestamp.get
}
else
{
- strOutput += "<font color=\"#A82F2F\"><font size=\"2\">" + dfTimestamp.format(alistContactTimestamp.get
+ strOutput += "<font color=\"#A82F2F\"><font size=\"2\">(" + dfTimestamp.format(alistContactTimestamp.get
}
}
@@ -535,10 +535,10 @@
private static void writeTextFile(String strFilename, String strFileContent)
{
// Date and Time
- DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd hh.mm.ss aa");
+ DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd.HHmmss");
Date currentDateTime = new Date();
- strFilename = "Table " + dateFormat.format(currentDateTime) + ".html";
+ strFilename = "" + dateFormat.format(currentDateTime) + ".html";
Writer writerTextFile = null;
RE: RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 10-03-2010 at 12:57 PM
quote:Ok, I will have to detect a specific valid email address instead of just parentheses: ( ).
Originally posted by ian_hawdon
Right, I've had a play with the script, there are a few bugs (for example, if an emoticon was used in a display name, the script can't find email addresses).
quote:I am aware that it is the current date, which is not what you want. That section of code was from an older program.
Originally posted by ian_hawdon
I have changed how the output is laid out, this makes Pidgin display the log correctly, but sadly, due to my lack of Java knowledge, I don't know how to make the file name show the date of the conversation, rather than the date of conversion.
quote:What software do you use to apply this kind of patch?
Originally posted by ian_hawdon
Here is the patch that needs to be applied to the Java source XP1 attached:
code:
...
RE: RE: RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 10-03-2010 at 01:08 PM
quote:
Originally posted by XP1
What software do you use to apply this kind of patch?
Ah, yeah, sorry, Linux user here
you'll need Patch, from here:
code:
http://gnuwin32.sourceforge.net/packages/patch.htm
copy the patch code into a file called "Main.patch" and save it in the same directory as Main.java
and run from command prompt:
code:
Patch Main.java < Main.patch
You can either add patch to your path, so it can be run form any folder, or just copy the patch.exe file to the same directory as your source code and run the command above
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 10-03-2010 at 03:47 PM
oh, and another bug I've stumbled upon: If the conversation wraps onto another line:
code:
[13:36:11] WANT A STABL: sorry about that (doing work on my phone
line!)
Then the script gets confused and crashes. So, possibly something in there needs to say if a line starts with " " then append to the line above.
I'm impressed with your script so far though
RE: RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 10-03-2010 at 05:20 PM
quote:A major change in version 1.01 is that I added file input. You can run through command line using the JAR file (...\dist\):
Originally posted by ian_hawdon
oh, and another bug I've stumbled upon: If the conversation wraps onto another line:
code:
[13:36:11] WANT A STABL: sorry about that (doing work on my phone
line!)
Then the script gets confused and crashes. So, possibly something in there needs to say if a line starts with " " then append to the line above.
I'm impressed with your script so far though
code:
java -jar "Messenger Plus! Legacy Chatlog.jar" SampleChatlog.txt
Can you give me an example of status change in both legacy and HTML?
How do you want me to handle the extra message lines? Do you want them to appear on a new line in HTML (<br>), or do you want them to be re-attached to the message so they become just a single line?
RE: Any way to convert legacy (.txt) logs into something useful? by CookieRevised on 10-03-2010 at 05:26 PM
PS: The reason why something like this hasn't been done before (at least not recently*) is that there are many many things you need to take into consideration when you want to convert a Plus! TXT log. As you have noticed, it is not as straitforward as one might think.
Apart from the few things/bugs you've already encountered, there are quite a lot more things which this script will probably 'crash' upon. And many of those things depend on user's settings at the time the log was created (also meaning it is very hard to get those settings so that the script is configured properly). Making it very hard to make an universal converter for stuff like this. That is: it is relativly easy to make a personal converter for one user, but it will probably not work for another user as-is.
* IIRC there once was a TXT converter posted on the forums though, many many years ago. But I'm not 100% sure of this.
RE: RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 10-05-2010 at 01:41 AM
quote:You're right. I was able to Google search for Messenger Plus! text chat logs (for research), and I have found a couple chat logs that people had posted publicly.
Originally posted by CookieRevised
PS: The reason why something like this hasn't been done before (at least not recently*) is that there are many many things you need to take into consideration when you want to convert a Plus! TXT log. As you have noticed, it is not as straitforward as one might think.
Apart from the few things/bugs you've already encountered, there are quite a lot more things which this script will probably 'crash' upon. And many of those things depend on user's settings at the time the log was created (also meaning it is very hard to get those settings so that the script is configured properly). Making it very hard to make an universal converter for stuff like this. That is: it is relativly easy to make a personal converter for one user, but it will probably not work for another user as-is.
For example, one chat log had "8 มกราคม 2548" for its "Session Start" in the header.
What kind of date is that? I have no idea.(Looks like the forums converted the characters to HTML code literally. You'll have to paste them into an HTML file or click on the Google search result for the date to view them.)
If someone donates some more examples or posts unexpected and expected results, maybe I could make it work better.
quote:I searched but didn't find any. If it exists, it would be cool if someone knows where it is.
Originally posted by CookieRevised
* IIRC there once was a TXT converter posted on the forums though, many many years ago. But I'm not 100% sure of this.
RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 10-05-2010 at 10:53 PM
Contact names should work with Unicode characters.
Are you interested in multiple contacts? The non-local users appear in brown color, though. Should there be other colors? Also, should the title have to be changed to include contacts > 2?
Try this one: 1.02.
It works with single-line and multiline messages, as well as status and action alerts.
Unless you're human, there's not really a way (for a computer) to know the difference between intended linebreak and unintended linebreak, so I just treat all multiline messages as linebreaks (HTML <br>) instead of losing them.
Unintended linebreak:
code:
tell me (call me on my home phone
line)!
Intended linebreak:
code:
Check this cool website out:
1) google.com
and this website is the same:
2) google.com
WOW!
Do you want status alerts to look like "SCREEN NAME STATUS.", with the status in uppercase? Like "Joe OUT TO LUNCH." or "Joe out to lunch." (with or without punctuation?)? "Joe OFFLINE" or "Joe offline" (with or without punctuation?)?
Check out "SampleChatlog (4).txt" for how it currently outputs status changes.
Some additional actions I know, which it can output to HTML:
code:Are there others?
[22:01:21] You have failed to receive file "LOL.lol" from LOL.
[16:50:14] You have successfully received C:\Documents and
Settings\LOL\My Documents\My Received Files\LOL.lol from LO
L.
[16:49:42] Cancel(Alt+Q)
[16:50:14] Open(Alt+P)
RE: Any way to convert legacy (.txt) logs into something useful? by CookieRevised on 10-06-2010 at 09:06 AM
quote:oh yes, I don't have a list of all of them, but there are many more (winks, webcam, video, files, sounds, .... have all messages for when you recieved, declined, failed, etc... and that is just from the top of my head)
Originally posted by XP1
Are there others?
PS: and they are, of course, all language dependant (the same goes for the datetime field btw, this also depends on the OS's local iirc).
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 10-07-2010 at 01:11 AM
Although I haven't tried converting one yet, I have a few conversations where someone changes their screen name during a conversation.
Do you need any examples of this happening?
I have to say, XP1, you're doing a great job, I really need to start looking into programming beyond VB!
RE: Any way to convert legacy (.txt) logs into something useful? by ian_hawdon on 12-02-2010 at 09:32 AM
New bug discovered:
If your screen name is too long, it will appear in the top part as "...<name here>", this won't match the conversation, meaning the whole conversation will be formatted in the other user's font.
RE: Any way to convert legacy (.txt) logs into something useful? by JimmyJazz on 03-24-2011 at 03:06 PM
Is there any instruction on how to use this? For example, where do I unpack the files, and how do I tell it which files to work on?
I'd love to get these old files into the HTML or XML format.
RE: RE: Any way to convert legacy (.txt) logs into something useful? by XP1 on 03-29-2011 at 04:05 PM
quote:Sorry, I have been away on this thread.
Originally posted by JimmyJazz
I'd love to get these old files into the HTML or XML format.
This would probably need a lot more improvements to meet your satisfaction, but if you want to try it out and see the output in ian_hawdon's HTML format, you will need to install the Java Runtime Environment (JRE), simply called Java as a shorthand.
Click here to see if you have Java installed:
http://www.java.com/en/download/installed.jsp?detect=jre&try=1
It doesn't really matter if you have an older version, but upgrades are recommended anyway.
Click here to download the latest version of Java:
http://www.java.com/en/download/manual.jsp
On your computer, after you install Java, you will need to extract the archive and then start a command prompt, shell, terminal, or console.
In Windows, you go to Start > Run > Enter "cmd" > Navigate to the JAR directory > Enter "java -jar "Messenger Plus! Legacy Chatlog.jar" SampleChatlog.txt".
Make sure your TXT file is in the same directory as the JAR file.