header
header Register : : Login header
header
divider
menuleft
menuright
submenu
left

[August 25th, 2008] Check the home page regarding PowerShell related news from a brand new sponsor: Idera

Converting IIS log files UTC time to local time
Last Post 22 Jan 2008 02:17 PM by kscriss. 4 Replies.
Printer Friendly
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
kscrissUser is Offline
Basic Member
Basic Member
Posts:119

--
18 Jan 2008 08:11 PM  

I am trying to convert IIS log files that are written in UTC time to our local time.

My input log file ex080117.log is 25,355KB, but my output file is 50,710KB.  The input file and the outuput file look identical when compared side-by-sed except that the time has been converted from UTC to our local time.

Why do you think the resulting output log size is so much larger than the input log size?

The scrpt is called by piping the input.log to the function and then piping the output from the function to a destination file.  The script is below.

# ls ex080117.log | ConvertIIS-LogFile | out-file 'my-logs.txt'
#
function ConvertIIS-LogFile
   {
      param( )
      begin { }
      process
         {
            $file = [System.IO.File]::OpenText( $_.fullname )
            while($line = $file.ReadLine())
              {    if(!$line.StartsWith('#') -and ($line.Length -gt 19))
                      {  $UTCdatestring = $line.Substring(0, 19)
                         $WorkDate = Get-Date($UTCdatestring)
                         $LocalWorkDate = $WorkDate.tolocaltime()
                         $FormatLocalWorkDate = $LocalWorkDate.ToString("yyyy-MM-dd HH:mm:ss";)
                         $StringLength = $line.length
                         $RestOfString = $line.substring(19,$StringLength - 19)
                         $OutString = "$FormatLocalWorkDate"+"$RestOfString"
                         Write-Output $OutString 
                      }
                   Else
                      {
                         Write-Output $line
                      }
              }
         }
      End {}
   }

My blog: http://blogs.powershellcentral.com/kscriss/
kscrissUser is Offline
Basic Member
Basic Member
Posts:119

--
18 Jan 2008 08:45 PM  
New mod: I'm converting the #DATE lines now as well. In the script above I was bypassing them. Don't spend too much time on my original brain-teaser because we figured out another way.

We are trying out the http://awstats.sourceforge.net/ Perl web site statistics suite for our enterprise's web site statistics. Its free and looks almost just like the one NetIQ is selling. Initially we couldn't figure out the TZ time zone variable. So we were going to convert all our logs timestamps from UTS to local time. We eventually broke down and read the awstats DOCs and figured out the TZ awstats TZ configuration thing. Now awsats is making the time conversions for us.

function ConvertIIS-LogFile
{
param( )
begin { }
process
{
$file = [System.IO.File]::OpenText( $_.fullname )
while($line = $file.ReadLine())
{ if(!$line.StartsWith('#') -and ($line.Length -gt 19))
{ $UTCdatestring = $line.Substring(0, 19)
$WorkDate = Get-Date($UTCdatestring)
$LocalWorkDate = $WorkDate.tolocaltime()
$FormatLocalWorkDate = $LocalWorkDate.ToString("yyyy-MM-dd HH:mm:ss")
$StringLength = $line.length
$RestOfString = $line.substring(19,$StringLength - 19)
# $RestOfString | Out-Host
$OutString = "$FormatLocalWorkDate"+"$RestOfString"
Write-Output $OutString
}
ElseIF ($line.contains("Date:"))
{
$UTCdatestring = $line.Substring(7, 19)
$WorkDate = Get-Date($UTCdatestring)
$LocalWorkDate = $WorkDate.tolocaltime()
$FormatLocalWorkDate = $LocalWorkDate.ToString("yyyy-MM-dd HH:mm:ss")
$BeginString = $line.substring(0,6)
$StringLength = $line.length
$RestOfString = $line.substring(7+19,$StringLength - 19 -7)
# $RestOfString | Out-Host
$OutString = "$BeginString"+"$FormatLocalWorkDate"+"$RestOfString"
Write-Output $OutString

}
Else
{
Write-Output $line
}
}
}
End {}
}
My blog: http://blogs.powershellcentral.com/kscriss/
bsonposhUser is Offline
Basic Member
Basic Member
Posts:393
Avatar

--
18 Jan 2008 08:57 PM  
I would seem to me that you could probably objectize the log in the same amount if not less lines. This would allow you do import them into a database as well as export/import xml files.
Brandon Shell
----------------
Microsoft Powershell MVP
https://mvp.support.microsoft.com/profile/Brandon
Blog: http://www.bsonposh.com
halr9000User is Offline
PowerShell MVP, Site Admin
Basic Member
Basic Member
Posts:335
Avatar

--
19 Jan 2008 02:07 AM  
Posted By kscriss on 01/18/2008 12:11 PM

My input log file ex080117.log is 25,355KB, but my output file is 50,710KB.  The input file and the outuput file look identical when compared side-by-sed except that the time has been converted from UTC to our local time.

Why do you think the resulting output log size is so much larger than the input log size?


I see you are using Out-File.  By default, it's text encoding is Unicode.  That means double the storage space over ASCII.  That probably explains your issue.  Be sure to specify -encoding ascii in your script.

Community Director, PowerShellCommunity.org
Co-host, PowerScripting Podcast
Author, TechProsaic
kscrissUser is Offline
Basic Member
Basic Member
Posts:119

--
22 Jan 2008 02:17 PM  
Ahh, I see. I'll give a try. 999,999+1 thank yous.
My blog: http://blogs.powershellcentral.com/kscriss/
You are not authorized to post a reply.

Active Forums 4.1
right
   
footer Sponsored by Quest Software • SAPIEN Technologies • ShellTools, LLC • Microsoft Windows Server 2008 footer
footer