jason_stephens
 New Member Posts:15

 |
| 03 Jun 2008 09:17 PM |
|
I have 2 shares on a server, X:\ and Y:\.
These 2 drives hold 2 terabytes each of what is supposed to be identical files and folders.
They don't appear to be the same. What I want to do is compare the two drives and list any anomalies.
thepowershellguy.com/blogs/posh/archive/2007/01/27/hey-powershell-guy-how-can-i-get-a-list-of-all-the-subfolders-in-a-folder-and-then-put-that-list-into-an-array.aspx
The above link demonstrates how to recursively list all folders and subfolders from a particular directory. Below is the actual code.
$dirs = dir -Recurse | Where {$_.psIsContainer -eq $true}
$dirs | select fullname
Now, with all that said, what would be the best way to compare these two drives? The above PS code does not show size of folders, but I would be satisfied with something that would just compare folder existence when compared.
I don't even know if all the folders and subfolders could fit into a PS array. Should I bring the both drives into arrays and then somehow compare the arrays? What suggestions could you guys scrounge up?
|
|
|
|
|
mhensley
 New Member Posts:14

 |
| 03 Jun 2008 09:30 PM |
|
Here's an outline (I don't have time to code it right now):
Loop through the first drive, adding each path\filename to a CSV file
Load the CSV file into a hash table (the key is the path\filename, the value can be anything you want to compare: size, mod date, etc).
Loop through the second drive, looking up each item in the hash table
If you find it, compare the size/moddate/whatever and report as desired; then DELETE it from the hash table
If you don't find it, report that the file only exists on the second drive
Loop through the hash table, reporting all remaining files as only existing on the first drive
Possible enhancement: if the list of files is too big for a hash table, add some sorting logic to process on folder at a time. |
|
|
|
|
bsonposh
 Basic Member Posts:392

 |
| 03 Jun 2008 11:55 PM |
|
This should work
$Source = "c:\data\psa"
$Dest = "c:\data\psa2"
# Create Hash Tables
$DriveX = @{}
$DriveY = @{}
# Fill Hash Tables
get-childitem $Source -rec | ?{!$_.PSisContainer} | %{$DriveX."$($_.FullName)" = $_.Length}
get-childitem $Dest -rec | ?{!$_.PSisContainer} | %{$DriveY."$($_.FullName)" = $_.Length}
# Check $DriveY with Elements from $DriveX
foreach($entry in $DriveX.Keys)
{
$name = $Entry -replace ($Source -replace "\\","\\"),$Dest
if($DriveX."$Entry" -eq $DriveY."$Name"){Write-Host "[$Name] and [$Entry] Are the Same [$Entry]"}
else{$Name}
} |
|
Brandon Shell ---------------- Microsoft Powershell MVP https://mvp.support.microsoft.com/profile/Brandon Blog: http://www.bsonposh.com |
|
|
bsonposh
 Basic Member Posts:392

 |
| 03 Jun 2008 11:57 PM |
|
You will need to change $Source and $Dest... Those are my test paths :) |
|
Brandon Shell ---------------- Microsoft Powershell MVP https://mvp.support.microsoft.com/profile/Brandon Blog: http://www.bsonposh.com |
|
|
jason_stephens
 New Member Posts:15

 |
| 04 Jun 2008 03:10 PM |
|
yeah :) I saw those. Working on the issue right now. Will respond with the end results and final code :)
|
|
|
|
|
halr9000 PowerShell MVP, Site Admin
 Basic Member Posts:334

 |
| 04 Jun 2008 04:07 PM |
|
Don Jones wanted to post but could not so he sent me this in email: Diff (dir ) (dir ) -property name Would probably do the job quite nicely. Might need to tweak the params a bit. |
|
Community Director, PowerShellCommunity.org Co-host, PowerScripting Podcast Author, TechProsaic |
|
|
bsonposh
 Basic Member Posts:392

 |
| 04 Jun 2008 05:26 PM |
|
wouldnt that only check the name? I didn't think about it, but perhaps you could do name and length |
|
Brandon Shell ---------------- Microsoft Powershell MVP https://mvp.support.microsoft.com/profile/Brandon Blog: http://www.bsonposh.com |
|
|
DonJ PowerShell MVP
 Basic Member Posts:134

 |
| 04 Jun 2008 05:31 PM |
|
I think the -property parameter of Compare-Object will accept multiple properties. Honestly, if the files are supposed to be IDENTICAL, you could just... Diff (dir x:\ -recurse) (dir y:\ -recurse) It'll compare every property of each file/folder by default, and show only differences. |
|
- Don Jones www.ConcentratedTech.com Subscribe (RSS) or visit for weekly PowerShell tips and lessons |
|
|
jason_stephens
 New Member Posts:15

 |
| 04 Jun 2008 05:51 PM |
|
Posted By DonJ on 06/04/2008 9:31 AM
I think the -property parameter of Compare-Object will accept multiple properties.
Honestly, if the files are supposed to be IDENTICAL, you could just...
Diff (dir x:\ -recurse) (dir y:\ -recurse)
It'll compare every property of each file/folder by default, and show only differences.
That is what I started to do but I ran out of memory trying to capture the first drives data. Guess I will be doing it a little at a time. Can you append to a file using export-csv?
|
|
|
|
|
bsonposh
 Basic Member Posts:392

 |
| 04 Jun 2008 05:57 PM |
|
No... AFAIK you cannot append. You could import append and then export. |
|
Brandon Shell ---------------- Microsoft Powershell MVP https://mvp.support.microsoft.com/profile/Brandon Blog: http://www.bsonposh.com |
|
|
jason_stephens
 New Member Posts:15

 |
| 04 Jun 2008 06:36 PM |
|
Trying the following now.
# Target locations to compare
$Target1 = Get-ChildItem "M:\" -name
$Target2 = Get-ChildItem "M:\" -name
Write-Host '$Target1 and $Target2 Comparing:'
Compare-Object -ReferenceObject $Target1 -DifferenceObject $Target2
Write-Host '$Target1 and $Target2 Compared.'
ForEach($SubFolder In $Target1)
{
$Folder1 = "M:\" + $SubFolder
$Folder2 = "N:\" + $SubFolder
Write-Host "$Folder1 and $Folder2 Comparing:"
Compare-Object -ReferenceObject $(Get-ChildItem -Recurse $Folder1) -DifferenceObject $(Get-ChildItem -Recurse $Folder2)
Write-Host "$Folder1 and $Folder2 Compared."
}
Write-Host "Done!"
Hope it works. Taking forever! :(
Please tell me that it is accepting those variables in there after my appends and I didn't monkey it up. I would hate to be waiting for this to be done only to find out that nothing is being compared or what I want isn't being compared. |
|
|
|
|
bsonposh
 Basic Member Posts:392

 |
| 04 Jun 2008 07:41 PM |
|
Depending on the quanty (not size) of files... most solutions could take a long time. |
|
Brandon Shell ---------------- Microsoft Powershell MVP https://mvp.support.microsoft.com/profile/Brandon Blog: http://www.bsonposh.com |
|
|
jason_stephens
 New Member Posts:15

 |
| 04 Jun 2008 07:54 PM |
|
Posted By bsonposh on 06/04/2008 11:41 AM
Depending on the quanty (not size) of files... most solutions could take a long time.
This is a large quantity of files. I help support an electronic medical record system at a hospital, umong other things. Eight-ish years of scanned images and text files.
The reason that I am trying to test this is because, unlike other system that require us to do replication for disaster recovery, this system writes the documents to 2 seperate locations on its own. The problem is that I don't think it did a perfect job. That is what I am trying to find out :)
Edit: Just noticed that my script is checking the M drive against the M drive at the top. That is ok. That particular piece has already been checked before. So, don't worry about it. The bottom half of the script is still right and that is what I am truely looking at anyway :) |
|
|
|
|
halr9000 PowerShell MVP, Site Admin
 Basic Member Posts:334

 |
| 04 Jun 2008 07:57 PM |
|
Since it takes so long, you may want to do a subset of the files if possible, that way you won't have wasted x hours. Put some logging into your script and leave it running for a while, check the log to make sure you see what you expected to. |
|
Community Director, PowerShellCommunity.org Co-host, PowerScripting Podcast Author, TechProsaic |
|
|
bruceatk
 New Member Posts:12

 |
| 07 Jun 2008 02:57 AM |
|
I would recommend using Robocopy to do the compare. You can capture the output with something like this: $FileCompare = robocopy dir1 dir2 /E /L The /L parm just logs a comparison. It doesn't do any copying. Using Powershell you can parse the output. "*Extra File"- File is in Dir2 but not Dir1 "New File" - File is in Dir1 but not Dir2 "Newer" - File is newer in Dir1 "Older" - File is newer in Dir 2 "Changed" - Same timestamp but different size Robocopy is reasonably fast at doing it without stressing your network, memory or cpu. It takes 9 minutes to compare 342,000 files and 22,757 directories across the network on my home server, while only using a a couple meg of memory. It would be much faster running on the same machine. Bruce |
|
|
|
|
halr9000 PowerShell MVP, Site Admin
 Basic Member Posts:334

 |
|
bruceatk
 New Member Posts:12

 |
| 07 Jun 2008 10:03 PM |
|
I think it's good to know both ways. There are times you want complete control in how it's done and PowerShell gives you that. If you need to get it done and Robocopy does what you need then do it that way. That is one of the things that I really like about PowerShell. If I need it quick, I can still use the same old command line tools I used before, but with the added benefit of being able to manipulate the output in a much more powerful and flexible way than I could before. I also think most people don't realize that Robocopy is now the recommended way to copy files. If you do XCopy /? in Vista you get this message: NOTE: Xcopy is now deprecated, please use Robocopy. Bruce |
|
|
|
|
halr9000 PowerShell MVP, Site Admin
 Basic Member Posts:334

 |
|