Wednesday, May 8, 2013

Dc agent troubleshoot automation with powershell

    Everyone can be tired from dealing with inconsistent data, especially when that particular data is a main part in performance report for executives. As a perfect example here can be any report from Websense Web Security containing users from Windows Active Directory.
    In Websense products Dc Agent is responsible for collecting and recording this piece of information. In details XidDcAgent table contains pairs of IP : Domain\Username, Timeout and Timestamps. But sometimes user has got new IP address long time ago, but somehow user map is showing old information. Or company has mixed policies based on IPs and user names and user is complaining that regular recourse is not accessible...
    
    Sounds familiar?

    For such troubleshooting Websense has console utility ConsoleClient, which allows you to export user map to file.

    Have you ever using this tool? ConsoleClient IP:Port and then - choose this option, press number, choose that, press number and so on.. Painful procedure, if you need to get exports from all your machines, in my case more than world wide 50 dc agents, and more than one hour to get only raw data. Yeah.

    Very good time for automation. As a result I want to have list of servers that have
problems (Number of entries in map is : 0, sounds very familiar too right?), list of servers with bad synchronization, list of outdated users. Knowing these results we can prevent many problems.

Total structure for our script will look like:

Namelist.txt – entry point, servers list.
Servername1:Port1
Servername2:Port2
#Servername3:Port3 - temporary exclude from check for maintenance reason.
Answers – folder where files of answers will be generated and stored for consoleclient utility.

Audit – folder with reports, which we will send to email after.

Problem_Servers – folder with list of problem servers.

Outdated-Servers – folder with list of outdated servers.

Scan_Result – outputs from consoleclient tool for each server.

As a beginning we need to create start.bat file. It will:

Showing status in window title ( believe me you don't want to see empty black screen in a middle of something)
Clean previous results
Generating file of answers for each server (answers in servername.txt from namelist.txt)
Running consoleclient with prepared answers.
Ps1.ps1 – powershell part to analyze raw data.
Ps2.ps1 – powershell part to send final result.



@set workdir1=c:\users\alehv\dcagent
@set diffhours=12


title Generating files with answers
@REM Creating file with answers...

erase %workdir1%\Answers\*.* /F /Q
erase %workdir1%\Audit\*.* /F /Q
erase %workdir1%\Outdated_Servers\*.* /F /Q
erase %workdir1%\Problem_Servers\*.* /F /Q
erase %workdir1%\Scan_Result\*.* /F /Q
@echo off
@for /F "eol=# tokens=1,2 delims=:" %%i in (%workdir1%\namelist.txt) do (

@if %%j==15869  ( 

@echo ^2> %workdir1%\Answers\%%i.txt
@echo ^1>> %workdir1\%Answers\%%i.txt
@echo ^3>> %workdir1\%Answers\%%i.txt
@echo %workdir1%\Scan_Result\%%i.log>>%workdir1%\Answers\%%i.txt
@echo ^5>> %workdir1%\Answers\%%i.txt
@echo q>> %workdir1%\Answers\%%i.txt 



@if %%j==30601  ( 

@echo ^2> %workdir1%\Answers\%%i.txt
@echo ^1>> %workdir1%\Answers\%%i.txt
@echo ^3>> %workdir1%\Answers\%%i.txt
@echo %workdir1%\Scan_Result\%%i.log>>%workdir1%\Answers\%%i.txt
@echo ^6>> %workdir1%\Answers\%%i.txt
@echo q>> %workdir1%\Answers\%%i.txt
)



title Gathering from: %%i
type %workdir1%\Answers\%%i.txt | "c:\program files (x86)\websense\web security\bin\"consoleclient.exe %%i %%j



)

@REM Data harvester
title Harvesting data...

powershell %workdir1%\ps1.ps1 %workdir1% %diffhours%

@REM  Send an email
title Sending an email...
powershell %workdir1%\ps2.ps1 %workdir1%

title Please check an email...


From Data harvester part we will have filled Answer folder ( Filtering service and Dc agent have different ports and answer numbers) and Scan_Result folder (servername.log with Ip/User records)

Ps1.ps1 will:
Move servername.log file to Problem_Servers if number of entries in map is equal zero
Move servername.log file to Outdated-Servers and rename to Servername.log has matches X from Total users when Timeout pass:Today minus DiffHours.
Collect all outdated users in Problem Client List.csv file
Save all information in html files in Audit folder
###########################################################################################
cls
$DiffHours=$args[1]
$Destination=$args[0]

Echo "Working Directory : $Destination"

Echo "Timeout           : $DiffHours"
Echo "___________________________________________________________"
###########################################################################################


function Get-TimeStamp-From-Line

{
Param ([string]$lin)

$keyphrase="Zzzz..."

$i=0
While ($keyphrase -ne "Timestamp:"){
$keyphrase= $lin.Split()[$i];
$i++
}
$lin=$lin.Split()[$i]+" "+$lin.Split()[$i+1]

Return $lin

}

###########################################################################################

function Get-TimeStamp-asDate
{
Param ([string]$l)


$l=$l.Substring(0, $l.IndexOf("."))

$ldate = [datetime]::ParseExact($l,"MM-dd-yyyy HH:mm:ss",$null) 

Return $ldate

}

###########################################################################################


$LazyClient=0


###########################################################################################

#### Moving all log files with Number of entries in map is : 0 to Problem_servers #### folder

(Get-ChildItem $Destination\Scan_result\ -Filter *.log  | Select-String "Number of entries in map is : 0") |  

Move-Item -Force -destination $destination\Problem_servers

###########################################################################################

#### Moving all log files with wrong dates to Outdated_servers folder

$xl = "$destination\Audit\Problem Client List.csv"

$xlOutput = '"Server"'+',"Client Time Stamp"'+',"Server Time Stamp"'+',"Client Details"'
echo  ($xlOutput >> $xl)

$fileEntries = [IO.Directory]::GetFiles("$Destination\Scan_Result\"); 

foreach($fileName in $fileEntries) 

    [Console]::WriteLine($fileName); 
$SnapShotTime=(get-content $fileName -totalcount 3)[-1] 
$SnapShotPart=$SnapShotTime.Split()[3]
$SnapShotTime=$SnapShotTime.Split()[2]
$TotalEntries=(get-content $fileName -totalcount 4)[-1] 
$TotalEntries=$TotalEntries.Split(":")[1]
$SnapShotPart=$SnapShotTime+" "+$SnapShotPart

$SnapShotPart=Get-TimeStamp-asDate($SnapShotPart)


$dataz = get-content $fileName

$SnapShotDate = "Timestamp: $SnapShotTime"
Echo  $SnapShotDate
#Echo  $SnapShotPart
#echo ($filename.Split('\')[-1] ).Split('.')[-2]
$PastRealValue = $SnapShotPart.AddHours(-$DiffHours)


for ($i=4; $i -le $dataz.length-2; $i++)

{

$TimeStampPart= Get-TimeStamp-asDate(Get-TimeStamp-From-Line($dataz[$i]))


$TimeStampValue=[datetime]::ParseExact($TimeStampPart,"MM/dd/yyyy HH:mm:ss",$null)


if ( $TimeStampValue -lt $PastRealValue){ 

if (($dataz[$i].Contains("Timeout: INFINITE")) -ne "True"){

$LazyClient=$LazyClient+1

$xlOutput='"'+(($filename.Split('\')[-1] ).Split('.')[-2])+'","'+$TimeStampValue+'","'+$SnapShotPart+'","'+$dataz[$i]+'"'
echo  ($xlOutput >> $xl)
}

}

}

Echo "Total outdated clients: $LazyClient from$TotalEntries"
If ($LazyClient -gt 0) { 


$filenameResult=$filename+" has matches - $LazyClient from$TotalEntries"

Move-Item -Force -path $filename -Destination $filenameResult 
Move-Item -Force -path $filenameResult -Destination $destination\Outdated_servers\

}


#Release counter for next server in list

$LazyClient=0
}       
Echo "___________________________________________________________"
###########################################################################################
####REPORTING PART
$date=get-date
$a = "<style>"
$a = $a + "BODY{background-color:peachpuff;}"
$a = $a + "TABLE{border-width: 1px;border-style: solid;border-color: black;border-collapse: collapse;}"
$a = $a + "TH{border-width: 1px;padding: 0px;border-style: solid;border-color: black;background-color:thistle}"
$a = $a + "TD{border-width: 1px;padding: 0px;border-style: solid;border-color: black;background-color:PaleGoldenrod}"
$a = $a + "</style>"
Get-ChildItem $destination\Outdated_Servers | ConvertTO-HTML Name -Title "Outdated Servers" -head $a -body "<H2>Outdated Servers: $date</H2>" | Set-Content $destination\Audit\Outdated.htm
Get-ChildItem $destination\Problem_Servers | ConvertTO-HTML Name -Title  "Problem Servers"  -head $a -body "<H2>Problem Servers:  $date</H2>"| Set-Content $destination\Audit\Problem.htm
Set-Content $xl -Encoding ASCII -Value (Get-Content $xl)

Ps2.ps1 will:
Send summary report with attached problem.html, outdated.html and problemclientlist.csv to email.

$Destination=$args[0]

$serveraddr=“emailserver.domain.com”

$date=get-date
$Toutdated=(get-ChildItem $Destination\Outdated_Servers).count
$Tproblem =(get-ChildItem $Destination\Problem_Servers).count

$MessageSubject = "DcAgents status report: $date"

$MessageBody = $Null
$MessageBody += "<head>"
$MessageBody += "<meta http-equiv=""Content-Type"" content=""UTF-8"" />"
$MessageBody += "</head>"
$MessageBody += "<body style=""font-family: 'Trebuchet MS';"">"
$MessageBody += "<div>Collected status report from DcAgents. <br><br>Report Stamp: $date.</div><br>"
$MessageBody += "<div>Total outdated servers : $Toutdated <br> Total problem servers  : $Tproblem</div><br>"
$MessageBody += "<div>Please check attached files.</div>"
$MessageBody += "</body>"

$Encoding = [System.Text.Encoding]::UTF8

[string[]]$Attachment = "$destination\Audit\outdated.htm", "$destination\Audit\problem.htm", "$destination\Audit\Problem Client List.csv"

$TO = "securityadministrator@domain.com" 


Send-MailMessage -SmtpServer $serveraddr -Encoding $Encoding -From "Websense@domain.com" -To $TO   -Subject $MessageSubject -BodyAsHtml $MessageBody -Attachment $Attachment
You may see, I'm not checking server availability, this responsibility has taken by monitoring server (topic for next article).
As usual any questions are welcome.




No comments:

Post a Comment