About pSearch

pSearch is a multi-search tool coded with Python, it is mainly focused on piracy related websites. It uses urllib and BeautifulSoup to connect and scrape the websites. SQLite3 is used to store the information about the websites.
This at first started as a practice for urllib and BeautifulSoup. After I finished their chapters from https://www.py4e.com/, I wanted to test them just to get a real experience of them. I also thought that, someone at some point will have to search for a specific software and game, and they have to go to the piracy sites individually one by one, so why not have a program that does it automatically? I know that there are a lot of other similar software and even websites, but for the sake of using urllib and BeautifulSoup, I wanted to do it.
It has evolved a lot from the first version. At first it was specifically several sites, then I thought that the program should be built in a way that doesn't need a lot of modifications when new sites are added. Hence, I added JSON for storing web information. Already at that point the program was automatic, every site I added didn't need configuration with the main program, only some have exceptions due to the links they put and the program scrapes. Then when I finished the Database chapter, I wanted to have an experience with that as well, just like with JSON. So now it works with SQLite, and with the SQLite Database Browser, it has become much simpler for other people to contribute to the project with new sites.

Database Content

• name: the name of the software.
• url: the main url of the website.
• searchurl: the url used for searching.
• key1_id, key2_id, key3_id:The program scrapes with three values (in most cases, there are some exceptions), the key1_id for example would have the value "div", the key2_id would have "class", and the key3_id would have the name that's inside the class. The program will use these three values in order to specifically find the area where the link (program's result) stands.
• type_id: this will specify the type of the website; games, software, android, movieseries, all.
• hasmainlink: some sites won't have the main link of the website in the a's href value. For example, they would just write /software-name.html, without the site's link at the first. The program would blow up if it didn't take care of it! So the program adds the main site's url at the beginning of the value that it scraped from the website.

All of these above can be viewed with the SQLite Browser using the websitesdb file.

Installing Requirements

BeautifulSoup is required for the application to run. The bs4 folder is included with the source code, just downloading it with pip is required from the user's end. To do this, you need to run the following command after having pip installed:
pip install -r requirements.txt
You can run this in the Command Prompt on Windows, and on a terminal of your choice on a Linux distribution. The command prompt/terminal should be opened within the folder that the requirements.txt file is included, you can run it normally but you have to identify the full directory/destination of the requirements.txt file instead of just writing "requirements.txt" at the end. "pip" is installed automatically once you install Python, which is necessary for running the application. Python can be installed from their official website: https://www.python.org/

Running pSearch

You can run the program by either opening a terminal/command prompt within the folder that pSearch.py is, or just normally but you have to include the full directory of the file:
python3 pSearch.py
OR
python pSearch.py
WITH DIRECTORY
python C:\Users\UserName\Documents\pSearch\pSearch.py

It should run without any problems if the requirements are installed.

You can get the exe from the Releases page on Github for faster running method, or by clicking the download for windows button.

Thank you for using the program!