• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
PythonForBeginners.com

PythonForBeginners.com

Learn By Example

  • Home
  • Python Tutorial
  • Python Basics
  • Python Code Examples

 

You are here: Home / Code / Python Code : Get all the links from a website

Python Code : Get all the links from a website

Last Updated: August 28, 2020

Overview


In this script, we are going to use the re module to get all links from any website. 

One of the most powerful function in the re module is "re.findall()".

While re.search() is used to find the first match for a pattern, re.findall() finds *all*
the matches and returns them as a list of strings, with each string representing one match.

Get all links from a website


This example will get all the links from any websites HTML code. 

To find all the links, we will in this example use the urllib2 module together
with the re.module
import urllib2
import re

#connect to a URL
website = urllib2.urlopen(url)

#read html code
html = website.read()

#use re.findall to get all the links
links = re.findall('"((http|ftp)s?://.*?)"', html)

print links

Happy scraping!

Recommended Python Training

For Python training, our top recommendation is DataCamp.

Free Trial

Filed Under: Code, Code Snippets, crawler, Python On The Web Date Originally Published: September 22, 2012

More Python Topics

API Basics Beautiful Soup bitly Cheatsheet Code Code Snippets Command Line crawler Data Types Development Dictionary Dictionary Data Structure In Python envoy Errorhandling Error Handling Exceptions Fabric Files fnmatch ftplib Games GUI Json Lists Loops Mechanzie Modules Modules In Python Mysql OS pil pip Python Python Code Snippets Python On The Web Python Strings Requests Scraping Scripts sh simplehttpserver System & OS urllib2 Web

Primary Sidebar

Get Our Free Guide To Learning Python

Menu

  • Python Basics
  • Code Examples
  • Loops
  • Functions
  • Strings
  • Python on the Web
  • Lists
  • Dictionaries
  • Python Modules
  • Python Glossary
  • Learn Python

Most Popular Content

  • Reading and Writing Files in Python
  • String Concatenation and Formatting
  • List Comprehensions in Python
  • How to use sys.argv in Python
  • How to use Split in Python
  • How to use comments in Python
  • Python Syntax Basics

Recent Posts

  • Datacamp Review 2020
  • Most Common Python Interview Questions For 2020
  • Python 2 Vs Python 3 with Examples
  • How To Run Your Python Scripts
  • The 5 Best Python IDE’s and Code Editors for 2019

Python Courses

  • Datacamp: Intro To Python
  • 2021 Complete Python Bootcamp
  • Python Mega Course: Build 10 Real World Apps
  • Python Data Science Bootcamp
  • Complete Python Developer: Zero to Mastery

Copyright © 2012–2021 ยท PythonForBeginners.com

  • Home
  • Contact Us
  • Privacy Policy
  • Write For Us