I am using element.locate & element .size to get location & size of element i want to click using pyautogui. Now problem is selenium gives coordinates in webpage whereas pyautogui calculate that coordinate on monitor's screen. So both coordinate are mismaching. If I find coordnate manually by . position () method in pyautogui than it works. But I first want to locate element via selenium and want to pass location of that element in pyautogui. Please help.
Trying to automate a usecase in Browserstack with selenium and python The below error was seen when trying to click on elements WebDriverException: Message: actions
Currently doing some scraping with Selenium and I am struggling with a page that seems to be fighting me:
The page opens a link in a new tab
I want to use driver.window_handles to iterate through the tabs to find the one I am looking for. However, this is somehow detected by the page and it closes the tab that I want to scrape as soon as I call window_handles :(
So I tried to disable the JavaScript closing function, using driver.execute_script() with window.close = function() { }; and close = function() { };. But this does not suppress the closing of the tab :(
I am really surprised as to what is going on here and how to bypass this.
Using selenium~=4.16.0. Also tried with undetected-chromedriver~=3.5.5.
Edit: just realized that disabling of the close function is of course not working. The call to close is in the new window/tab and I don't think I can override it from the old tab :(
Hey Everyone, I wrote a program that iterates through each airbnb listing, gets data and moves on the next listing. The problem is it just loops one listings. any ideas?
def fetch_images_from_carousel(driver): images = set() print("Fetching images from Carousel:") try: print("Loading images") WebDriverWait(driver, 10).until( EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".itu7ddv.atm_e2_idpfg4.atm_vy_idpfg4.atm_mk_stnw88.atm_e2_1osqo2v__1lzdix4.atm_vy_1osqo2v__1lzdix4.i1cqnm0r.atm_jp_pyzg9w.atm_jr_nyqth1.i1de1kle.atm_vh_yfq0k3.dir.dir-ltr")) ) initial_images = driver.find_elements(By.CSS_SELECTOR, ".itu7ddv.atm_e2_idpfg4.atm_vy_idpfg4.atm_mk_stnw88.atm_e2_1osqo2v__1lzdix4.atm_vy_1osqo2v__1lzdix4.i1cqnm0r.atm_jp_pyzg9w.atm_jr_nyqth1.i1de1kle.atm_vh_yfq0k3.dir.dir-ltr") for img in initial_images: src = img.get_attribute('src') if src and isinstance(src, str) and src not in images: # Ensure src is a string and not previously added images.add(src) print("Added image:", src)
# Handle the carousel navigation using JavaScript to click the button try: while True: next_button = driver.find_element(By.CSS_SELECTOR, ".ms83rji.atm_fg_esu3gu.dir.dir-ltr") driver.execute_script("arguments[0].click();", next_button) # Click the button using JavaScript time.sleep(3) # Adjust the delay to allow the next set of images to load (e.g., 3 seconds) new_images = fetch_current_visible_images(driver) print(f"Fetched {len(new_images)} new images after clicking the button.") print("New Images:", new_images) if set(new_images) == images: logging.info("No new images loaded. Exiting carousel navigation.") break except TimeoutException: logging.info("Timed out waiting for carousel elements.") except Exception as e: logging.error(f"Error navigating carousel: {str(e)}")
return images
heres a portion of the code that fails. The main is
# Find listings one by one to avoid stale element references while True: listings = WebDriverWait(driver, 10).until( EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".dir.dir-ltr")) )
for listing in listings: try: # Perform scraping operations for each listing images = fetch_images_from_carousel(driver) price = listing.find_element(By.CSS_SELECTOR, "._1jo4hgw").text beds = listing.find_element(By.CSS_SELECTOR, ".a8jt5op.atm_3f_idpfg4.atm_7h_hxbz6r.atm_7i_ysn8ba.atm_e2_t94yts" ".atm_ks_zryt35.atm_l8_idpfg4.atm_mk_stnw88.atm_vv_1q9ccgz.atm_vy_t94yts.dir" ".dir-ltr").text ratings = listing.find_element(By.CSS_SELECTOR, ".t1a9j9y7.atm_da_1ko3t4y.atm_dm_kb7nvz.atm_fg_h9n0ih.dir.dir-ltr").text
# Click on the listing to view more details if needed listing.click()
# Wait for the new page to load WebDriverWait(driver, 10).until(EC.url_changes(url))
listing_id = driver.current_url.split('/')[-1] # Extract listing ID from URL insert_or_update_listing_data(table_name, listing_id, images, price, beds, ratings)
# Go back to the previous page to continue scraping other listings driver.back() except NoSuchElementException: continue except StaleElementReferenceException: continue # Check if there's a next page try: next_page_button = driver.find_element(By.CSS_SELECTOR, "._1u3e89e") next_page_button.click() except NoSuchElementException: break except Exception as e: logging.error(f"Error processing a listing: {e}")
I need help... I am using a python program to send parameters to the Spain e-administration site to get an appointment for a procedure. But the page is blocking my session: The requested URL was rejected. Please consult with your administrator.
Your support ID is: <1417176221212705327608>. .
I was reading some solutions, and I saw that using sleep.time() I could simulate the writing of a human so that the page doesn't crash. But it didn't work. Can anyone think of something else?
I made a bot that logs into my facebook daily, checks to see whose birthday it is via the events page, and submits a happy birthday message via the form field. Facebook seems like it’s always changing, so this bot has changed a lot over the years. Most recently, once the birthday form fields are identified by XPATH, an exception occurs indicating that send_keys can’t type inside the form (can’t interact with the element). I eventually got this to work with ActionChains, but the send_keys method it uses can’t seem to handle emojis. Is there another way to click inside the form and send the keys?
On macOS Ventura (version 13.4.1) with an M2 chip with amr64 architecture, I`m trying to run tests by using PyCharm 2023 Community edition, with selenium robot framework 6.11, Python 3.11, and up to date web driver manager installed with Chrome browser (version 15.0.5790.170 (Official Build) (arm64) ) and compatible version of chrome web driver(amr64). After running a test getting the following error : OSError: [Errno 86] Bad CPU type in executable: '/Users/igormelnyk/PycharmProjects/python/selenium_mini_course/venv/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager' Can anybody Clarify? Thanks. Spent about 3 hours navigating GPT, checked all versions for compatibility, Official source for Chromium and Selenium, retyped scripts in Python, reactivated the virtual environment, pip all the libraires and....... Still nothing....
Hello everyone, I'm new to selenium and also not so deep into the coding at all so I'm quite happy to got so far :D
So I want to automate a Process where I need to klick a Button, this Button get's only active after a few seconds. This is because of a script that got implemented to see if everything is okay to proceed. I placed the script at the End here.
My Problem is now, that I can't click the button with selenium.
I tried these options:
and none of them worked, I think it's because the element is part of the script and not a normal button in the source code. Does anyone here have an idea how to solve this problem?
<div class="navbar navbar-expand-md navbar-light action-bar"> <div id="action-toolbar" name="action-toolbar"> </div><script>kendo.syncReady(function(){jQuery("#action-toolbar").kendoToolBar({"items":[{"enable":false,"id":"save-button","text":"\u003ci class=\u0027icon-floppy-disk\u0027\u003e\u003c/i\u003eSpeichern","type":"button","attributes":{"class":"k-primary save-button","data-activewhendirty":"","role":"button","aria-disabled":"true","disabled":"disabled","tabindex":"-1"}},{"enable":false,"id":"reset-button","text":"\u003ci class=\u0027icon-undo\u0027\u003e\u003c/i\u003eVerwerfen","type":"button","attributes":{"class":"reset-button","data-activeif":"ContractChangeDetails.resettable","role":"button","aria-disabled":"true","disabled":"disabled","tabindex":"-1"}},{"enable":false,"id":"totalValueCalculationButton","text":"\u003ci class=\u0027icon-calculator3\u0027\u003e\u003c/i\u003eGesamtwertberechnung","type":"button","attributes":{"data-id":"1008429","role":"button","data-activeif":"ContractChangeDetails.calculatable","disabled":"disabled"},"click":ContractChangeDetails.totalValueCalculationClickHandler},{"enable":false,"id":"setContractPrice","text":"\u003ci class=\u0027icon-coin-euro\u0027\u003e\u003c/i\u003eVertragspreis setzen","type":"button","attributes":{"data-id":"1008429","role":"button","data-activeif":"ContractChangeDetails.calculated","disabled":"disabled"},"click":ContractChangeDetails.setContractPriceClickHandler},{"enable":false,"id":"transferToContract","text":"\u003ci class=\u0027icon-add-to-list\u0027\u003e\u003c/i\u003eSofort in den Vertrag überführen","type":"button","attributes":{"data-id":"1008429","data-activeif":"ContractChangeDetails.transferable","role":"button","disabled":"disabled"},"click":ContractChangeDetails.transferToContractClickHandler},{"enable":false,"id":"discardContractChange","text":"\u003cspan class=\u0027k-icon k-i-x-outline\u0027\u003e\u003c/span\u003eVertragsänderung verwerfen","type":"button","attributes":{"data-id":"1008429","data-activeif":"ContractChangeDetails.discardable","role":"button","disabled":"disabled"},"click":ContractChangeDetails.discardClickHandler},{"enable":false,"id":"createKonkalProcess","text":"\u003ci class=\u0027icon-link\u0027\u003e\u003c/i\u003eVorgang anlegen","type":"button","attributes":{"data-id":"1008429","data-activeif":"ContractChangeDetails.createProcessIsAllowed","role":"button","disabled":"disabled"},"click":ContractChangeDetails.konkalCreateProcessClickHandler},{"enable":true,"id":"export-button","text":"\u003ci class=\u0027icon-download\u0027\u003e\u003c/i\u003eExportieren \u003ci class=\u0027icon-arrow-down22\u0027\u003e\u003c/i\u003e","type":"button","attributes":{"data-activeif":"ContractChangeDetails.exportable","role":"button"}},{"enable":false,"id":"import-button","template":jQuery('#fileUploadDialogTemplate').html(),"text":"File","type":"button","attributes":{"data-id":"1008429","data-contextmenu":"false"}}]});});</script> </div>
P.s.: it's an intranet website, so I can't share the link with you.
How do I instruct selenium to use the beta version of chrome? (not chromedriver - it is finding that - but chrome itself). This is using selenium and python on linux (ubuntu 22.04).
I'm trying to use chrome 116 (which is /bin/google-chrome-beta on my system). I've already installed chromedriver 116, but when it launches it still attempts to drive the older version of chrome (115), not the beta version of chrome (116), which only results in a fatal error indicating that chromedriver 116 cannot be used with chrome 115.
I tried to replace /bin/google-chrome with /bin/google-chrome-beta but that breaks it completely.
There's a selenium chrome option for path to the executable, but that is apparently deprecated. (I didn't try it)
Big goal is I would like to make program that able to scrape the URL of the websites from my read later list on chrome.
I figure the first step is to get selenium to click on the 3 dot menu button on chromes toolbar. But this is part of the browser and not the webpage and when I try to right click to inspect it for its elements nothing happens.
How do I get selenium to interact with the browsers toolbar buttons?
This tool would help greatly as I am a highschooler that needs to take the SAT in June but since all the spots near me were full I keep checking the website everyday to find test centers that might’ve opened up but even when they do, they fill up too quickly for me to sign up so this tool helps check for open ones so I don’t have to stay on the website 24/7. Any help or redirects to somewhere I can ask for help would be awesome, thanks!
I am trying to automate a process, with a excel file sended with Microsoft Encryption. At end of process after a I check my account. Show up a page without XPath, and a just need click on a button. Somebody know how to deal with pages with autoid?
At first it was version error so I upgraded selenium, updated chrome Version 116.0.5845.111, and got a new webdriver Version 116.0.5845.96. versions are not exact, is that a problem? there is no webdriver with the same version otherwise.
Also the error went away but I think there is a problem with finding the path since it never gets passed
I'm looking to scrape a website once an hour just for basic data. Is this too often that could trigger a captcha or would it be case by case depending on the website?
I am new to coding, I get the gist behind it but I get lost when it comes to debugging. I keep arriving at a InvalidArguementException when using the driver.get() function. There are multiple variables and lists involved which I will show in the actual code below.
But to fully understand my intention is to scrape Linkedin by using google to filter my search results and have the chromewebdriver access and log each link google provides into a pandas dataframe.
linkedin_urls = [my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='yuRUbf']")))]
Lnks.append(linkedin_urls)
for x in Lnks: for i in x:
# get the profile URL
print(i)
driver.get(i)
sleep(randint(3,5))
Please let me know of any suggestions or if more info is required. Again, I am a complete noob at coding so some technical aspects may go over my head. Thank you.
How ever i can not manage to click the "Anmelden" Button because there is no ID or something i can search for...(maybe additional info: the button is changing the cullor if you put the mouse on it)
This is how the html is looking:
Anyone having a suggestion what I can do? Is there a working solution to work with such "hidden" buttons?