반응형
Notice
Recent Posts
Recent Comments
Link
관리 메뉴

bro's coding

NLP.Sentence Tokenization 본문

[AI]/NLP

NLP.Sentence Tokenization

givemebro 2021. 2. 3. 14:43
반응형
from nltk.tokenize import sent_tokenize
text="Local farmers are protesting strengthened regulations on living shelters for foreign laborers introduced in the wake of a Cambodian worker’s death, claiming the measures are ill-devised and demanding more support from the government.In a press conference Tuesday near the Ministry of the Employment and Labor building in Sejong, the Korean Advanced Farmers Federation demanded the government retract the measure that stipulates improved shelter conditions for foreign workers.“Chaos is growing across agricultural sites as the government started enforcing the changed measure without much time to prepare,” the federation said in a statement.“It is inevitable for farms employing foreign workers to change their farming plans, and much damage is expected for the sector due to decreased output and delays from difficulty in securing manpower.”The Labor Ministry announced in December that it will reject employment permit applications by employers that offer unsuitable shelters starting Jan. 1 in response to the death of the Cambodian worker."
print(sent_tokenize(text))

'''
['Local farmers are protesting strengthened regulations on living shelters for foreign laborers introduced in the wake of a Cambodian worker’s death, claiming the measures are ill-devised and demanding more support from the government.In a press conference Tuesday near the Ministry of the Employment and Labor building in Sejong, the Korean Advanced Farmers Federation demanded the government retract the measure that stipulates improved shelter conditions for foreign workers.“Chaos is growing across agricultural sites as the government started enforcing the changed measure without much time to prepare,” the federation said in a statement.“It is inevitable for farms employing foreign workers to change their farming plans, and much damage is expected for the sector due to decreased output and delays from difficulty in securing manpower.”The Labor Ministry announced in December that it will reject employment permit applications by employers that offer unsuitable shelters starting Jan. 1 in response to the death of the Cambodian worker.']
'''

 

 

 

반응형

'[AI] > NLP' 카테고리의 다른 글

ChatBot.DialogFlow  (0) 2021.03.02
NLP.한국어 문장 토큰화.KSS(Korean Sentence Splitter)  (0) 2021.02.03
NLP.Word Tokenization  (0) 2021.02.03
Comments