r/SeleniumPython • u/webscrapingpro • Dec 08 '23
Python Selenium Tutorial #13 - Proxies Explained: How to Use Them Effectively
r/SeleniumPython • u/webscrapingpro • Dec 08 '23
r/SeleniumPython • u/QuietBlaze • Nov 27 '23
I can find innumerable tutorials; I don't want those.
I want to go to a web-based API reference for the Python selenium module (or whatever passes for that), where I can lookup, e.g., webdriver.Remote() and see every possible argument that can be passed to that method.
Can anyone point me in the correct direction?
Many thanks!
r/SeleniumPython • u/Realistic-Page-1665 • Nov 18 '23
Hello everyone, I’m trying to use Selenium to find block of text 1 and block of text 2, but it gives an error that the text was not found. I specified it using a selector and xpath, but still doesn’t see it.
I am attaching the code and a picture of what you need to find
# Функция для поиска элемента по селектору ID
def find_element_by_id(driver, id):
try:
return WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, id))
)
except TimeoutException:
return None
# Функция для сравнения текста из двух элементов
def compare_text(text1, text2):
# Преобразуем текст в нижний регистр, чтобы сделать сравнение более точным
text1 = text1.lower()
text2 = text2.lower()
# Используем метод `similarity()` Bard API для сравнения двух текстов
similarity = bard.similarity(text1, text2)
# Возвращаем значение, указывающее, подходит ли текст
return similarity >= 0.8
# Находим элемент 1
element_1 = find_element_by_id(driver, "#klecks-app > tui-root > tui-dropdown-host > div > task > flex-view > flex-common-view > div.tui-container.tui-container_adaptive.flex-common-view__main > div > main > flex-element > flex-container > flex-element:nth-child(1)")
# Находим элемент 2
element_2 = find_element_by_id(driver, "#klecks-app > tui-root > tui-dropdown-host > div > task > flex-view > flex-common-view > div.tui-container.tui-container_adaptive.flex-common-view__main > div > main > flex-element > flex-container > flex-element:nth-child(4)")
# Получаем текст из элементов
text1 = element_1.text
text2 = element_2.text
# Сравниваем текст
is_similar = compare_text(text1, text2)
# Выводим результат сравнения
if is_similar:
result = "Тексты похожи"
else:
result = "Тексты не похожи"
print(result)

r/SeleniumPython • u/[deleted] • Nov 15 '23
As the title says guys...
Im looping my Selenium script 200 times with the cmd "times".
Is there another command I can place at the end of my code that will count and show me live how often the script has looped already?
Thanks for your help!
r/SeleniumPython • u/Straight_Molasses891 • Oct 31 '23
hello to all interested. I have been looking for a solution to my question for more than a month and have tried more than one option. I have my own tradingview strategy, its results depend on the selection of parameters. I want to find a way to automate the selection of parameters to find the most profitable combination. I used chat gpt to write code in python, it even worked as it should. but there was a problem with access to ccxt libraries for a maximum of 2-5 days depending on the exchange. now I am thinking about whether it is possible to solve my problem with the help of selenium.
anyone with a similar experience would love to hear your thoughts
r/SeleniumPython • u/Puzzleheaded-Tie5827 • Oct 22 '23
r/SeleniumPython • u/Apprehensive-Dirt419 • Oct 19 '23
Hello everyone , I am using Selenium Python to scrape a website https://www.whed.net/results_institutions.php
and This website contains data for every country(list of institutions) and one need to click on link of every instituion and scrape name , location and www associated with that institution.
Now I have tried to use selenium for automating the task and I am doing mostly fine other than being unable to close the dialog box.
This is my sample code. Can somebody Explain me how to do it.
service = Service("C:/Selenium_drivers/chromedriver-win64/chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get(url)
country = 'Afghanistan'
institues = []
cities = []
wwws = []
drop_down = Select(driver.find_element(By.XPATH, '//select'))
drop_down.select_by_visible_text(country)
all_institute = driver.find_element(By.XPATH, "//input[@id='membre2']")
if not all_institute.is_selected():
all_institute.click()
button = driver.find_element(By.XPATH, "//input[@type='button']")
button.click()
results_per_page = Select(driver.find_element(By.XPATH, "//select[@name='nbr_ref_pge']"))
results_per_page.select_by_visible_text('100')
total_results = int(driver.find_element(By.XPATH, "//p[@class='infos']").text.split()[0])
max_iter = total_results//100 + 1
iterations = 0
go_on = True
while go_on:
iterations += 1
institutions = driver.find_elements(By.XPATH, "//li[contains(@class, 'clearfix plus')]")
for institue in institutions:
link = institute.find_element(By.XPATH, ".//h3/a")
link.click()
time.sleep(2)
pop_up = driver.find_element(By.XPATH, "//iframe[starts-with(@id, 'fancybox-frame')]")
driver.switch_to_frame(pop_up)
# main_window = driver.current_window_handle # Store the handle of the main window
# popup_window = None
# for window_handle in driver.window_handles:
# if window_handle != main_window:
# popup_window = window_handle
# Switch to the popup window
# driver.switch_to.window(popup_window)
institue = driver.find_element(By.XPATH, "//div[@class='detail_right']/div[1]").text
city = driver.find_element(By.XPATH, "//span[@class='libelle' and text() = 'City:']/following-sibling::span[@class='contenu']").text
www = driver.find_element(By.XPATH, "//span[@class='libelle' and text() = 'WWW:']/following-sibling::span[@class='contenu']").get_attribute("title")
institues.append(institute)
cities.append(city)
wwws.append(www)
close_button = wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Close']")))
close_button.click()
# driver.switch_to.window(main_window)
# driver.switch_to.window(main_window)
if iterations >= max_iter:
go_on =False
break
time.sleep(2)
next_page = driver.find_elements(By.XPATH, "//a[@title='Next page' ]")[0]
next_page.click()r/SeleniumPython • u/A-Nit619 • Oct 04 '23
Hey guys....I am using selenium python to automate ERP system but there is a 2F authentication which stops my automation. The 2F authentication usually requires the user to enter the SMS code they receive on their phone. Please let me know how I can bypass this?
r/SeleniumPython • u/cryptosage • Sep 27 '23
Hi all, first time poster here. Been building a scraper for the past few days, but I am completely stuck on how to differentiate between pricing of options for the same product in my CSV. I can pull ALL of the pricing from the menu of items page, but then it will get them out of order in the python lists because pricing for one item with variations/options will end up taking up multiple spaces in the list, and make them match up with the wrong indexes of the item name/brand/etc.
Also, I can't find anything unique and in a pattern via ID, CLASS or XPATH to break the options' pricing up into their own list items.
Can anybody help?
r/SeleniumPython • u/Shot-Craft-650 • Sep 26 '23
I am trying to access this website: "www.realestate.com.au"
I can access it normally using my normal browser but I am not able to access it using:
Requests (I accessed it in the start but now it is causing issues Error Code: 429)
Selenium (It does not show anything. White blank page)
Undetected Chrome Driver (It does not show anything. White blank page)
Can anybody check this website and tell me how can I access it and get some data from it?
r/SeleniumPython • u/DMVTECHGUY • Sep 23 '23
Does anybody know how to get rid of these?
r/SeleniumPython • u/abd_rzk • Sep 20 '23
Why when i login in my instagram with selenium the "Not Now' button doesn't work
error : selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:{"method":"xpath","selector":"//*[@id="mount_0_0_LY"]/div/div/div[2]/div/div/div/div[1]/div[1]/div[1]/div/']
r/SeleniumPython • u/[deleted] • Sep 14 '23
r/SeleniumPython • u/Bobydude967 • Sep 13 '23
r/SeleniumPython • u/TileJam209 • Sep 05 '23
I am new to web scraping and Python. I am trying to scrape the sector performance chart on https://digital.fidelity.com/prgw/digital/research/sector . I know how to use the basics of selenium. When I use a chrome driver to find an element by xpath, for example,
driver.find_element(By.XPATH, value = '//*[@id="market-sector-performance-table"]/tbody/tr[1]/td[2]') to get the S&P 500 performance for 1 month (-1.59%), it says could not find element with that xpath. On some websites that I scrape, this works, on others like the Fidelity site, it doesn't. Why is this? Does it have anything to do with JavaScript or that the website is dynamical? What is the work-around to get the elements I need in this case?
Similarly, I have tried using BeautifulSoup to get the data, and I get empty lists with no data, or errors saying elements could not be found.
How specifically would I scrape the Fidelity chart with Python? Specific code would be very helpful.
r/SeleniumPython • u/[deleted] • Sep 04 '23
r/SeleniumPython • u/zero_opacity • Aug 30 '23
I’m looking for a sample project with a basic test suite that I could use when testing how to set up and configure Selenium Grid in a CI/CD pipeline. Is anyone aware of like a starter project I could just use vs writing my own?
r/SeleniumPython • u/Divided_By_Zeroo • Aug 24 '23
This pop-up cannot be inspected, I need to click on 'Allow' & 'Cancel'.
I've tried using pyautogui (2 Tabs, then Enter), and even though that works, it may not always as the script can run into scenarios where the browser is minimized.
(As mentioned before, I am unable to inspect this pop-up content using Dev-tools or any other way)
Thanks in advance.
r/SeleniumPython • u/[deleted] • Aug 24 '23
r/SeleniumPython • u/Big_Cadoe • Aug 23 '23
Selenium (or chrome webdriver) stopped working.
At first it was version error so I upgraded selenium, updated chrome Version 116.0.5845.111, and got a new webdriver Version 116.0.5845.96. versions are not exact, is that a problem? there is no webdriver with the same version otherwise.
Also the error went away but I think there is a problem with finding the path since it never gets passed
driver = webdriver.Chrome(path)
Assuming im getting the path correct, are there any concerns I should have?
Thanks in advanced,
r/SeleniumPython • u/warrobe • Aug 17 '23
Hi everyone! This morning I found out that the latest chromedriver version on https://chromedriver.chromium.org/downloads: 114 is not compatible with my current browser version 116.
Does that mean I have to wait until an updated version of the chromedriver becomes available? Many thanks in advance!
r/SeleniumPython • u/frink42 • Aug 17 '23
How do I instruct selenium to use the beta version of chrome? (not chromedriver - it is finding that - but chrome itself). This is using selenium and python on linux (ubuntu 22.04).
I'm trying to use chrome 116 (which is /bin/google-chrome-beta on my system). I've already installed chromedriver 116, but when it launches it still attempts to drive the older version of chrome (115), not the beta version of chrome (116), which only results in a fatal error indicating that chromedriver 116 cannot be used with chrome 115.
I tried to replace /bin/google-chrome with /bin/google-chrome-beta but that breaks it completely.
There's a selenium chrome option for path to the executable, but that is apparently deprecated. (I didn't try it)
r/SeleniumPython • u/CranberryFar6635 • Aug 11 '23
On macOS Ventura (version 13.4.1) with an M2 chip with amr64 architecture, I`m trying to run tests by using PyCharm 2023 Community edition, with selenium robot framework 6.11, Python 3.11, and up to date web driver manager installed with Chrome browser (version 15.0.5790.170 (Official Build) (arm64) ) and compatible version of chrome web driver(amr64).
After running a test getting the following error :
OSError: [Errno 86] Bad CPU type in executable: '/Users/igormelnyk/PycharmProjects/python/selenium_mini_course/venv/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager'
Can anybody Clarify? Thanks.
Spent about 3 hours navigating GPT, checked all versions for compatibility, Official source for Chromium and Selenium, retyped scripts in Python, reactivated the virtual environment, pip all the libraires and....... Still nothing....