How to use Python to send URL requests to the server without actually opening the browser (“Do not use WebBrowser Module”)?

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa %2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194"

data=urllib.urlencode({"login":"your login or whatever, "pass":"password }) # be aware you need to change "login" and "pass" to names of fields in form you have.
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2 .HTTPCookieProcessor(jar))
request = urllib2.Request("url for authentication", data)
opener.open(request) # now you should be authorized and able to send any request like logged in user , using opener

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily -dollar-store-1349194%3Flid%3D1349194"
anything = opener.open(url)
anything.read()

I want to This URL is sent to the server as a request to change the content on the website when I log in. The problem is that when I open the URL with mechanize or urllib2, it does not change anything on the website. However, when I use the webbrowser module , It does change the content on the website. I want to do the function of the webbrowser module, but the actual browser is not opened. Is there a way to do this? Why are mechanization and urllib2 not working?

Edit: What I mean by “changes to the website” is that these things I get are called “shares” and “tickets” and are used for the information I put on the website. My The program finds the accurate information (if it is fake, they will kick you out) and use the URL to “insert” it into the website.

Sample URL (all of my others follow this format) :

http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp $JspView $SaveAction and amplifier; inPlaceID = 1020634218& xxx_c_1_f_987 ​​= HTTP: //www.yellowpages .com/pittsburgh-pa/mip/family-dollar-store-1349194 cover = 1349194

Mechanization code:

import mechanize
br = mechanize.Browser()
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com% 2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
br.open(url)

urllib2 code:

from urllib2 import urlopen
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh -pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
page = urllib2.urlopen(url)
page.read()

webbrowser code:

import webbrowser
url = http: //www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%1942Ffamily-dollar-store-1349%lar-store 3Flid%3D1349194
webbrowser.open(url)

Edit#2
I just tried this code:

import urllib2 
import urllib

def log_in():
 url ='https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction'
 values ​​= {'inUserName':'[email protected]',
'inUserPass':'myPass'}
 data = urllib.urlencode(values)
 req = urllib2.Request (url, data)
 req.add_header('Host','www.locationary.com')
 req.add_header('User-Agent','Mozilla/5.0 (Windows NT 6.1; rv: 8.0) Gecko/20100101 Firefox/8.0')
 req.add_header('Accept','text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q= 0.8')
 req.add_header('Accept-Language','en-us,en;q=0.5')
 req.add_header('Accept-Encoding','gzip, deflate')< br /> req.add_header('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7')
 req.add_header('Connection','keep- alive')
 req.add_header('Referer','http://www.locationary.com/')
 req.add_header('Cookie','site_version=REGULAR; __utma=47547066.1079503560.1321924193.1322707232 .1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; locaCountry=1033; locaState=1795; locaCity=Montreal; jforumUserId=1; PMS=1 ; TurnOFfTips=true; Locacookie=enable; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; PMS=1; __utmb501; =47547066.15.10.1324693472; __utmc=47547066; JSESSIONID=DC7F5AB08264A51FBCDB836393CB16E7; PSESSIONID=28b334905ab6305f7a7fe051e83857bc280af1a9; __utmc=47547066; __utmb=47547066.15. 10.1324693472; ACTION_RESULT_CODE=ACTION_RESULT_FAIL; ACTION_ERROR_TEXT=java.lang.NullPointerException')
 req.add_header('Content-Type','application/x-www-form-urlencoded')
 response = urllib2.urlopen (req)
 page = response.read()

url2 ='http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987 =http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194'

log_in()
response2 = urllib2. urlopen(url2)
page2 = response2.read()

But it does not work.

Edit 3: The code from tony does not work for me.

p>

import urllib2
import urllib
import cookielib

data = urllib.urlencode({"inUserName":"MYUSERNAMESHOULDBEHERE", " inUserPass":"MYPASSWORDSHOULDBEHERE"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request(" https://www.locationary.com/index.j sp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
opener.open(request) 
url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$ JspView$SaveAction&inPlaceID=1012432546&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Fdennys-13470813%3Flid%3D13470813"
anything = opener.open(url)
anything. read()

Final edit!
I finally used Tony’s suggestion to make it work!

This is my final code:

import urllib2
import urllib
import cookielib

 data = urllib.urlencode({"inUserName":"[email protected]", "inUserPass":"mypassword"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2. build_opener(urllib2.HTTPCookieProcessor(jar))
opener.addheaders.append(('User-agent','Mozilla/4.0'))
opener.addheaders.append( ('Referer','http ://www.hellboundhackers.org/index.php') )
opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz=47547066.1324655802.52.13.utmcsr =google|utmccn=(organic)|utmcmd=organic|utmctr=cache:dr23PN5fUj4J:www.locationary.com/%20locationary; nickname=jacob501; jforumUserId=1; PMS=1; locaCountry=1033; locaState=1786; locaCity= Vancouver; JSESSIONID=A8F241E1924CE7A25FAA8C5CA6597697; PSESSIONID=5c21c44245f978b917f17982c944a9ec2b5d2df5; Locacookie=enable; __utmb=47547066.5.10.1324930160; __utmc=47547066'))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
response = opener.open(request) 
url = " http://www.locationary.com/"
anything = opener.open(url)
anything.read()

All I have to do is add lines

opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz=

and so on (really long code OK, cookie)

I also added a “Referer” and “User-Agent” header just in case.

Thank you!

First you should write the url variable in quotes:

url = "http:/ /www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%1942Ffamily-dol3-larid-1 %3D1349194"

If you want to send a request without opening the browser, you can use urllib, just like you.

If you need authentication (it seems to be this ), you should send an authentication request, get cookies (use it as cookielib.FileCookieJar) and set them in the opener. Then, you will be able to open the page and send the request.

Probably you need something like this :

data=urllib.urlencode({"login":"your login or whatever, "pass":"password}) # be aware you need to change "login" and "pass" to names of fields in form you have.
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request("url for authentication", data)
opener.open(request) # now you should be authorized and able to send any request like logged in user, using opener

url = "http://www.locationary.com/access/proxy.jsp?ACTI ON_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194 = opener.openthing%3D1349194"
 url)
anything.read()