header
header Register : : Login header
header
divider
menuleft
menuright
submenu
left

[August 25th, 2008] Check the home page regarding PowerShell related news from a brand new sponsor: Idera

Subject: capture URL redirect in variable
Prev Next
You are not authorized to post a reply.

Author Messages
clay202User is Offline
New Member
New Member
Posts:4

03/07/2008 10:05 PM  

Hello all,  I'm a PowerShell newbie -- trying to leverage the technology at my company.  I got a question about an issue I hit and I wanted to see if anyone have an idea on how I can solve it.

Statement of Work: I have got a database full of URL redirects that I need to test (for example, I will connect to the old URL http://www.mycompany.com/support/test.page and our web site will redirect to http://www.mycompany.com/newsupport/newpage.html .  Since the list is long and I want to continue to build my PowerShell skills, I want to use PowerShell to test each redirect.  

Issue: I can use PowerShell to retrieve a single web page and read the wewb page, have flow control, and that (Thanks PowerShell in Action Book!).  What I can't figure out if there is a way to capture the new URL.  For example, the steps I want to take:

- connect to the URL in the database using .NET System.Net.WebClient.DownloadString and dump into variable.  For example, I call www.xyz.com/support/page1.html but a URL redirect happens and I am now on www.xyz.com/techsupport/page2.html, how can I retrieve the newly redirected URL?  Would it be in my variable object?  if so, whats its name?  I tried browsing MSDN but I cannot seem to locate it (I'm hopefully that it exists).

If somebody could provide syntax on how I can retrieve that newly redirected URL, that would be great.  Please keep in mind, I'm a newbie... if you could provide the answer as speaking to a beginner , that would be great.  Thanks in advance for your response.

halr9000User is Offline
CLI Addict
CLI Addict
Posts:245


03/08/2008 8:21 AM  

The webclient class is easy but limited in some ways.  There's several ways to grab stuff off the net, and some are better than others, depending on what you want to do.  I had a discussion with a guy on the net about this a while ago, and here is the blog post he wrote in response.

Check out the below code for an example using system.net.HttpWebRequest.  Note that the URL I started with is one thing (halr9000.com/feed) and the one returned in the ResponseUri property is another (feedburner.com).  Comparing the two should be a good method for you to use to solve this problem.  It sounds like you are the type who will want to try to work this out yourself.  Let us know how that goes and if you need another prod, just ask.  Then be sure to post your answer so others can benefit.


37# $url = "http://halr9000.com/feed"
38# $webreq = [System.Net.HttpWebRequest]::Create($url)
39# $response = $webreq.GetResponse()
40# $response.StatusCode
OK
41# $response


IsMutuallyAuthenticated : False
Cookies : {}
Headers : {X-FB-Host, Connection, Content-Type, Date, ETag, Last-Modified
erver}
ContentLength : -1
ContentEncoding :
ContentType : text/xml;charset=utf-8
CharacterSet : utf-8
Server : Apache
LastModified : 3/7/2008 11:11:02 AM
StatusCode : OK
StatusDescription : OK
ProtocolVersion : 1.0
ResponseUri : http://feeds.feedburner.com/halr9000
Method : GET
IsFromCache : False



42# $response.Close()

Community Director, PowerShellCommunity.org
Co-host, PowerScripting Podcast (http://powerscripting.net)
Author, TechProsaic (http://halr9000.com)
halr9000User is Offline
CLI Addict
CLI Addict
Posts:245


03/08/2008 8:25 AM  
P.S. this question will be covered in the next powerscripting podcast.

Community Director, PowerShellCommunity.org
Co-host, PowerScripting Podcast (http://powerscripting.net)
Author, TechProsaic (http://halr9000.com)
clay202User is Offline
New Member
New Member
Posts:4

03/10/2008 8:47 AM  

halr9000, thank you for your reply.  I will test it out this week.  Your response is what I am looking for.

When I have tested out my code, I will post it on here.  I won't be able to work on it until the middle of the week but I have put a reminder in my calender to respond by Friday 3/14.

 

clay202User is Offline
New Member
New Member
Posts:4

03/12/2008 8:42 PM  

Hello all,

Here is the script I created to find the URL when a redirect happens (I will feed it an old URL, a redirect will be performed and I would grab the new URL)

Thanks in advance for assistance.

# This script performs the following function:
#
# (1) read a row (foreach statement) from file input.txt (see row format beleow)
# (2) Split up the row and place the URL into a variable
# (3) using the URL, use HttpWebRequest to retrieve the web page and store in object
# (4) the object will have the newURL (the redirect). 
# (5) write the row with new URL to a new file
#
# input.txt file row layout looks like...
#     www.mycompany.com/support/page5.htm,{JAKSJ-KAKS21-...}
#
$b=Get-Content output.txt
foreach ($row in $b) {
 $e=@{}
 $e.url, $e.cmsguid = $row.split("`,")
 $webreq = [System.Net.HttpWebRequest]::Create($oldurl)
 $response = $webreq.GetResponse()
 $redirecturl = $response.ResponseUri.AbsoluteUri
 write-host "New URL: " $redirecturl
 $response.Close()
 $row = $row + "," + $redirecturl
 echo $row >> newoutput.txt
 Write-Host "`n`n Old: $oldurl ----maps to--->  $redirecturl `n`n"

}

halr9000User is Offline
CLI Addict
CLI Addict
Posts:245


03/14/2008 7:01 AM  
Nice. Any reason you didn't use impot-csv on the output.txt file?

Community Director, PowerShellCommunity.org
Co-host, PowerScripting Podcast (http://powerscripting.net)
Author, TechProsaic (http://halr9000.com)
clay202User is Offline
New Member
New Member
Posts:4

03/14/2008 5:03 PM  

Actually, no reason... I could of used Import-CSV.

Thanks again for your assistance.

 

halr9000User is Offline
CLI Addict
CLI Addict
Posts:245


03/14/2008 5:21 PM  
I actually like the technique you did use, at least for other scenarios. Not many people know you can do multiple assignments to hash tables that way.

Community Director, PowerShellCommunity.org
Co-host, PowerScripting Podcast (http://powerscripting.net)
Author, TechProsaic (http://halr9000.com)
You are not authorized to post a reply.
Forums > Using PowerShell > General PowerShell > capture URL redirect in variable



ActiveForums 3.7
right
   
footer Sponsored by Quest Software • SAPIEN Technologies • ShellTools, LLC • Microsoft Windows Server 2008 footer
footer