I am using Beautiful Soup to parse webpages and printing the name of the webpages visited on the terminal. However, often the name of the webpage has single left (\u2018) and right(\u2019) character which the python can't print as it gives charmap encoding error. Is there any way to remove these characters?

3

1 Answer

These codes are Unicode for the single left and right quote characters. You can replace them with their ASCII equivalent which Python shouldn't have any problem printing on your system:

>>> print u"\u2018Hi\u2019" ‘Hi’ >>> print u"\u2018Hi\u2019".replace(u"\u2018", "'").replace(u"\u2019", "'") 'Hi'

Alternatively with regex:

import re s = u"\u2018Hi\u2019" >>> print re.sub(u"(\u2018|\u2019)", "'", s) 'Hi'

However Python shouldn't have any problem printing the Unicode version of these as well. It's possible that you are using str() somewhere which will try to convert your unicode to ascii and throw your exception.

1

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password

Post as a guest

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy