How to use codes

[salmankhan302]

Body

Dear All,

I am a newbie and trying to use some of the existing codes for web scraping. I want to use the below python code for web scraping. I put it in Jupyter's Note book but it is not working there. Can somebody suggests to me which notebook should I use to run below webscraping program.
https://github.com/salmankhan302/Scraping-TripAdvisor-with-Python-2020

BR
Salman Saleem

Guidelines

• I have read and understood this category's guidelines before making this post.

[salmankhan302]

when i run it at jupyter notebook it shows below error
ModuleNotFoundError Traceback (most recent call last)
Cell In[1], line 3
1 import sys
2 import csv
----> 3 from selenium import webdriver
4 import time
6 # default path to file to store data

ModuleNotFoundError: No module named 'selenium'

[airtower-luna]

ModuleNotFoundError: No module named 'selenium'

That means the selenium module isn't installed where you're running the code. See the installation instructions in the Selenium repository.

You don't get the same error for the other modules in your code (sys, csv, time) because they are part of the Python standard library, so you always have them if you install Python. 😉

[salmankhan302]

Many thanks Now it is working 🥇

[salmankhan302]

Another problem can someone help me how to fix below issue
time.sleep(2)
---> 39 driver.find_element_by_xpath("//span[@Class='taLnk ulBlueLinks']").click()
41 container = driver.find_elements_by_xpath(".//div[@Class='review-container']")
43 for j in range(len(container)):

AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath'

[airtower-luna]

That looks like you're following a heavily outdated tutorial or example code. The find_element_by_* and find_elements_by_* functions were removed in Selenium 4.3.0 (and deprecated a while before that), the current version is 4.9.0. You should switch to the current find_element/find_elements functions, that take a parameter that says what kind of thing you're looking for.

For example the line where you got the error could be fixed like this:

from selenium.webdriver.common.by import By
# [...]
driver.find_element(By.XPATH, "//span[@Class='taLnk ulBlueLinks']").click()

See also: https://www.selenium.dev/documentation/webdriver/elements/locators/#xpath

[salmankhan302]

Many thanks and here is another one. Please help me I am a few steps away to run this
ElementClickInterceptedException Traceback (most recent call last)
Cell In[61], line 40
38 time.sleep(2)
39 from selenium.webdriver.common.by import By
---> 40 driver.find_element(By.XPATH, "//span[@Class='taLnk ulBlueLinks']").click()
41 #driver.find_element_by_xpath("//span[@Class='taLnk ulBlueLinks']").click()
42 container = driver.find_elements_by_xpath(".//div[@Class='review-container']")

File ~\AppData\Local\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py:94, in WebElement.click(self)
92 def click(self) -> None:
93 """Clicks the element."""
---> 94 self._execute(Command.CLICK_ELEMENT)

File ~\AppData\Local\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py:403, in WebElement._execute(self, command, params)
401 params = {}
402 params["id"] = self._id
--> 403 return self._parent.execute(command, params)

File ~\AppData\Local\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py:440, in WebDriver.execute(self, driver_command, params)
438 response = self.command_executor.execute(driver_command, params)
439 if response:
--> 440 self.error_handler.check_response(response)
441 response["value"] = self._unwrap_value(response.get("value", None))
442 return response

File ~\AppData\Local\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py:245, in ErrorHandler.check_response(self, response)
243 alert_text = value["alert"].get("text")
244 raise exception_class(message, screen, stacktrace, alert_text) # type: ignore[call-arg] # mypy is not smart enough here
--> 245 raise exception_class(message, screen, stacktrace)

ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (600, 523). Other element would receive the click:

(Session info: MicrosoftEdge=112.0.1722.64)
Stacktrace:
Backtrace:
GetHandleVerifier [0x00007FF6FBFFDB12+60274]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBF8E772+818882]
(No symbol) [0x00007FF6FBC0DFAE]
(No symbol) [0x00007FF6FBC58573]
(No symbol) [0x00007FF6FBC55868]
(No symbol) [0x00007FF6FBC52C77]
(No symbol) [0x00007FF6FBC516DA]
(No symbol) [0x00007FF6FBC43076]
(No symbol) [0x00007FF6FBC6F56A]
(No symbol) [0x00007FF6FBC42751]
(No symbol) [0x00007FF6FBC42621]
(No symbol) [0x00007FF6FBC6F8A0]
(No symbol) [0x00007FF6FBC42751]
(No symbol) [0x00007FF6FBC8891C]
(No symbol) [0x00007FF6FBC6F343]
(No symbol) [0x00007FF6FBC41796]
(No symbol) [0x00007FF6FBC40975]
(No symbol) [0x00007FF6FBC41F04]
Microsoft::Applications::Events::EventProperties::SetLevel [0x00007FF6FBE96BF7+1676711]
Microsoft::Applications::Events::EventProperties::SetLevel [0x00007FF6FBD3EEBD+268397]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBECF907+36951]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBEC7A15+4453]
Microsoft::Applications::Events::ILogManager::DispatchEventBroadcast [0x00007FF6FC1C7EB3+1319011]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBF9682C+851836]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBF92894+835556]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBF929EC+835900]
Microsoft::Applications::Events::EventProperty::~EventProperty [0x00007FF6FBF885C1+793873]
BaseThreadInitThunk [0x00007FFF30E17604+20]
RtlUserThreadStart [0x00007FFF322226A1+33]

[airtower-luna]

Small thing first: You should not put the import of By with the find_element function call, that goes into the header with your other imports.

As for the exception, the browser you're controlling doesn't seem to like clicking that element. I can't say why, I don't use Edge. 🦊

What I can say is:

• Clicking a span element seems odd at best. Unless the site you're accessing is really badly made clickable elements should be ones with the proper semantics (e.g. a, button, ...). Maybe you actually need another element.
• If you need help with the Edge problem, it's best to make a separate post, or even look at the Selenium community.
• When posting code or stack traces, make sure to use a code block. It's hard to read otherwise.