In fact, it is not necessary to use in my crawling process, but this time I want to use prettify() function of it to print out a clear layout and perform regular expression processing. sudo apt-get install chromium-driverĪfter the installation is complete, install BeautifulSoup4 package. pip3 install seleniumĪnd we need to install chromium driver. I can set the keywords to be queried and the number of pages I want to crawl, and then print out the title and url of query results.įirst, we need to install selenium and webdriver_manager. Today’s note is based on the Google Search Engine and Python + Selenium. often used tools or some knowledge.īecause I have done a lot of crawling work now, but I am afraid that this part of the work will stop for a while, and I am afraid that my crawling related skills will become unfamiliar. For a long time, I want to write an article about crawlers, recording the regular expression, IP settings, user agent… etc.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |