데이터 마이닝, 정보 검색 등과 같은 다양한 도메인에서 웹 페이지의 콘텐츠를 추출할 수 있습니다. 신문 및 잡지의 웹 사이트에서 정보를 추출하기 위해 신문 라이브러리를 사용할 것입니다.
이 라이브러리의 주요 목적은 신문 및 유사한 웹사이트에서 기사를 추출하고 선별하는 것입니다.
설치:
-
신문 라이브러리를 설치하려면 터미널에서 실행하십시오:
$ pip install newspaper3k
-
lxml 종속성의 경우 터미널에서 아래 명령을 실행하십시오.
$pip install lxml
-
PIL을 설치하려면 실행
$pip install Pillow
-
NLP 말뭉치가 다운로드됩니다.
$ curl https://raw.githubusercontent.com/codelucas/newspaper/master/download_corpora.py | python
파이썬 newpaper 라이브러리는 기사와 관련된 정보를 수집하는 데 사용됩니다. 여기에는 저자 이름, 기사의 주요 이미지, 발행 날짜, 기사에 있는 비디오, 기사를 설명하는 키워드 및 기사 요약이 포함됩니다.
#Import required library from newspaper import Article # url link-which you want to extract url = "https://www.wsj.com/articles/lawmakers-to-resume-stalled-border-security-talks-11549901117" # Download the article >>> from newspaper import Article >>> url = "https://www.wsj.com/articles/lawmakers-to-resume-stalled-border-security-talks-11549901117" >>> article = Article(url) >>> article.download() # Parse the article and fetch authors name >>> article.parse() >>> print(article.authors)
출력:
['Kristina Peterson', 'Andrew Duehren', 'Natalie Andrews', 'Kristina.Peterson Wsj.Com', 'Andrew.Duehren Wsj.Com', 'Natalie.Andrews Wsj.Com']
# Extract Publication date
>>> print("Article Publication Date:")
>>> print(article.publish_date)
# Extract URL of the major images
>>> print(article.top_image) 출력:
https://images.wsj.net/im-51122/social
# Extract keywords using NLP
print ("Keywords in the article", article.keywords)
# Extract summary of the article
print("Article Summary", article.summary) 아래는 전체 프로그램입니다:
from newspaper import Article
url = "https://www.wsj.com/articles/lawmakers-to-resume-stalled-border-security-talks-11549901117"
article = Article(url)
article.download()
article.parse()
print(article.authors)
print("Article Publication Date:")
print(article.publish_date)
print("Major Image in the article:")
print(article.top_image)
article.nlp()
print ("Keywords in the article")
print(article.keywords)
print("Article Summary")
print(article.summary) 출력:
['Kristina Peterson', 'Andrew Duehren', 'Natalie Andrews', 'Kristina.Peterson Wsj.Com', 'Andrew.Duehren Wsj.Com', 'Natalie.Andrews Wsj.Com'] Article Publication Date: None Major Image in the article: https://images.wsj.net/im-51122/social Keywords in the article ['state', 'spending', 'sweeping', 'southern', 'security', 'border', 'principle', 'lawmakers', 'avoid', 'shutdown', 'reach', 'weekendthe', 'fund', 'trump', 'union', 'agreement', 'wall'] Article Summary President Trump made the case in his State of the Union address for the construction of a wall along the southern U.S. border, calling it a “moral issue." Photo: GettyWASHINGTON—Senior lawmakers said Monday night they had reached an agreement in principle on a sweeping deal to end a monthslong fight over border security and avoid a partial government shutdown this weekend. The top four lawmakers on the House and Senate Appropriations Committees emerged after three closed-door meetings Monday and announced that they had agreed to a framework for all seven spending bills whose funding expires at 12:01 a.m. Saturday.