Needed to do this today, as I’d set up a FAST Search crawl of a  public-facing website, and there being a lot of errors I need to do some serious analysis of the issues.  Of course, there is no “Export to Excel” tab, which would have been great. Still after hunting around and deciding that the Codeplex project which was written for SharePoint 2007 probably wouldn’t work (though I never actually confirmed this), as I’m assuming things like DLL versions will be a bit off, though being codeplex, you could of course recompile with the latest DLLs.

Better still, “how about a bit of Powershell?” I thought, seeing as I’m getting over my fear of that beastie and sure enough there is an excellent blog post from the SharePoint escalation team which gives a great example of how to export the Crawl Log.  I took the liberty of pinching that code and altering it slightly to output to a CSV text file, so it goes stright into Excel nicely. Here’s the code:

#============================================================================ 
#Powershell script to pull all the crawl logs and display based on errorId 
#============================================================================ [IO.Directory]::SetCurrentDirectory((Convert-Path (Get-Location -PSProvider FileSystem)))
#Replace "Search Service Application" in the script with the exact name of the SSA that you browse to for viewing the crawl log. 
#With FAST you have multiple Search SSA’s and hence specify the name of the SSA that you use to view the crawl log data. 
$ssa = Get-SPEnterpriseSearchServiceApplication | Where-Object {$_.Name -eq "UoN FAST Search Connector Service Application"} 
#This should list only one SSA object. 
$ssa 
#Create a LogViewer object associated with that SSA 
$logViewer = New-Object Microsoft.Office.Server.Search.Administration.Logviewer $ssa 
#Get a List of all errors/warnings in the Crawl Log
$ErrorList = $logViewer.GetAllStatusMessages() | Select ErrorId 
#Loop through each type of error and pull that data $currentUser = [Environment]::UserDomainName + "\" + [Environment]::UserName;
$machine = [Environment]::MachineName;
$date = ( get-date ).ToString('yyyyMMdd');
$logFile = "$date-installationlog.txt";
if ($logfile -eq $null)
{
  $logFile = New-Item -type file "$date-installationlog.txt"; 
}
Foreach ($errorId in $ErrorList) 
{ 
    $crawlLogFilters = New-Object Microsoft.Office.Server.Search.Administration.CrawlLogFilters 
    #Filter based on the Error Id 
    $crawlLogFilters.AddFilter("MessageId", $errorId.errorId)        
    "Pulling data for Message ID : " + $errorId.errorId 
    $nextStart = 0 
    $urls = $logViewer.GetCurrentCrawlLogData($crawlLogFilters, ([ref] $nextStart)) 
    #Data from the crawl log will be available in the DataTable $urls. If this number is larger than the number of records requested (50 by default), then use only 50 records, ignore the rest. 
    WHILE($nextStart -ne -1)
 {
  $crawlLogFilters.AddFilter("StartAt", $nextStart);
  $nextStart = 0;
  $urls = $logViewer.GetCurrentCrawlLogData($crawlLogFilters, ([ref] $nextStart));

  for ($i=0; $i -le $urls.Rows.Count -1 ; $i++)
  {
   # Just output the URL, the Error (if any) and the Error Code.
   [System.String]::Format('{0},{1}, {2}',$urls.Rows[$i].DisplayUrl, $urls.Rows[$i].ErrorMsg, $urls.Rows[$i].ErrorLevel) | Out-File $logFile -append;
  } 
 } 
}

Note that the LogViewer class is deprecated, but I’m assuming something will replace it.  Hope this helps somebody

Cheers

Dave Mc