Updated: Monday, October 16,2023-10-16 12:27:27

main
Shwetha Jayaraj 2023-10-16 12:27:29 -04:00
parent f78156d0d2
commit 422a0f2f94
5 changed files with 37 additions and 8 deletions

View File

@ -1,5 +1,13 @@
{
"recentFiles": [
{
"basename": "Webscraping",
"path": "Coding Tips (Classical)/Terminal Tips/GUIs/Tools/Webscraping.md"
},
{
"basename": "Robots.txt Files",
"path": "Robots.txt Files.md"
},
{
"basename": "Potentiometers & Analog SerialReader",
"path": "Machine Tips (Quantum)/Physics/Hardware/Potentiometers & Analog SerialReader.md"
@ -46,7 +54,7 @@
},
{
"basename": "Fire",
"path": "Coding Tips (Classical)/Terminal Tips/Fire.md"
"path": "Coding Tips (Classical)/Terminal Tips/Languages/Python/Fire.md"
},
{
"basename": "Choosing a Name for Your Computer",

View File

@ -18,8 +18,21 @@
"source": false
}
}
},
{
"id": "0a0de85a51848b9d",
"type": "leaf",
"state": {
"type": "markdown",
"state": {
"file": "Coding Tips (Classical)/Terminal Tips/GUIs/Tools/Webscraping.md",
"mode": "source",
"source": false
}
}
}
]
],
"currentTab": 1
}
],
"direction": "vertical"
@ -78,7 +91,7 @@
}
],
"direction": "horizontal",
"width": 200
"width": 323.5
},
"right": {
"id": "4bd9c02fbfe6785f",
@ -94,6 +107,7 @@
"state": {
"type": "backlink",
"state": {
"file": "Coding Tips (Classical)/Terminal Tips/GUIs/Tools/Webscraping.md",
"collapseAll": false,
"extraContext": false,
"sortOrder": "alphabetical",
@ -110,6 +124,7 @@
"state": {
"type": "outgoing-link",
"state": {
"file": "Coding Tips (Classical)/Terminal Tips/GUIs/Tools/Webscraping.md",
"linksCollapsed": false,
"unlinkedCollapsed": true
}
@ -132,7 +147,7 @@
"state": {
"type": "outline",
"state": {
"file": "Excalidraw/Drawing 2023-10-16 12.13.42.excalidraw.md"
"file": "Coding Tips (Classical)/Terminal Tips/GUIs/Tools/Webscraping.md"
}
}
}
@ -161,8 +176,9 @@
},
"active": "dbad7b010371d947",
"lastOpenFiles": [
"Machine Tips (Quantum)/Physics/Hardware/Potentiometers & Analog SerialReader.md",
"Robots.txt Files.md",
"Excalidraw/Drawing 2023-10-16 12.13.42.excalidraw.md",
"Machine Tips (Quantum)/Physics/Hardware/Potentiometers & Analog SerialReader.md",
"Excalidraw",
"Machine Tips (Quantum)/Physics/Optics.md",
"Machine Tips (Quantum)/Physics/Hardware/Laser Cutting.md",
@ -174,7 +190,7 @@
"Machine Tips (Quantum)/Math/Proof of a formula for modulo.md",
"Machine Tips (Quantum)/Math/Quantum Formalism.md",
"Machine Tips (Quantum)/Math/Visualizing the Quantum Space.md",
"Coding Tips (Classical)/Terminal Tips/Fire.md",
"Coding Tips (Classical)/Terminal Tips/Languages/Python/Fire.md",
"Coding Tips (Classical)/Terminal Tips/Computers/Choosing a Name for Your Computer.md",
"Coding Tips (Classical)/Terminal Tips/CLI Tools/Terminal Emulators.md",
"Coding Tips (Classical)/Terminal Tips/CLI Tools/fzf.md",
@ -191,7 +207,6 @@
"Coding Tips (Classical)/Project Vault/Current Occupations/Manhattan Youth",
"Coding Tips (Classical)/Project Vault/Current Occupations/Website Projects/My Domain Names.md",
"Coding Tips (Classical)/Project Vault/Current Occupations/Potential and Future/Career Tips.md",
"Coding Tips (Classical)/Project Vault/Current Occupations/Potential and Future/60 careers/Academic Tutor.md",
"Coding Tips (Classical)/Project Vault/About Obsidian/imgFiles/Pasted image 20231011091043.png",
"Coding Tips (Classical)/Project Vault/About Obsidian/Slides & Tools/export/Slides/plugin/chalkboard/_style.css",
"Coding Tips (Classical)/Project Vault/About Obsidian/Slides & Tools/export/Slides/plugin/chalkboard/img/blackboard.png",

View File

@ -3,7 +3,7 @@
Webscraping is a common task in the CS world that makes it easy and efficient to extract large amounts of data. It is part of a larger topic of data mining which allows for the human understandable analysis of all the data that is out there.
You will often use requests and beautifulsoup libraries.
You will often use requests and beautifulsoup libraries. To prevent webscraping on your own sites, refer to the rob
---

View File

@ -0,0 +1,6 @@
Robots.txt is an increasingly important file found on websites that determine whether you permit a website crawler to index your page for search engine optimization. As webscraping is entirely legal in the US, this is the wild west of scraping and thus I want to keep mu brain and information safe from scraping.
*Resources*:
- [Robots.txt file examples](https://blog.hubspot.com/marketing/robots-txt-file)