Skip to content

Commit 2c73105

Browse files
committed
add documentation for decoding newlines in the io module
1 parent dcf3cc5 commit 2c73105

File tree

1 file changed

+47
-2
lines changed

1 file changed

+47
-2
lines changed

Doc/library/io.rst

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1065,6 +1065,13 @@ Text I/O
10651065
.. versionchanged:: 3.11
10661066
The method supports ``encoding="locale"`` option.
10671067

1068+
Example usage::
1069+
1070+
# decoding newlines in universal newlines mode
1071+
>>> import io
1072+
>>> io.TextIOWrapper(io.BytesIO(b'1\r\n2\r3\n4')).read()
1073+
'1\n2\n3\n4'
1074+
10681075
.. method:: seek(cookie, whence=os.SEEK_SET, /)
10691076

10701077
Set the stream position.
@@ -1139,15 +1146,53 @@ Text I/O
11391146
# .getvalue() will now raise an exception.
11401147
output.close()
11411148

1149+
# decoding newlines in universal newlines mode
1150+
io.StringIO('1\r\n2\r3\n4', newline=None).getvalue()
1151+
# '1\n2\n3\n4'
1152+
11421153

11431154
.. index::
11441155
single: universal newlines; io.IncrementalNewlineDecoder class
11451156

1146-
.. class:: IncrementalNewlineDecoder
1157+
.. class:: IncrementalNewlineDecoder(decoder, translate, errors='strict')
11471158

11481159
A helper codec that decodes newlines for :term:`universal newlines` mode.
1149-
It inherits from :class:`codecs.IncrementalDecoder`.
1160+
It inherits from :class:`codecs.IncrementalDecoder`. It also records the types
1161+
of newlines encountered. When used with ``translate=False``, it ensures
1162+
that the newline sequence is returned in one piece. If an incremental
1163+
decoder is not required, :class:`TextIOWrapper` or :class:`StringIO`
1164+
should be used.
1165+
1166+
*decoder* is an incremental decoder or ``None``. If set to ``None`` it
1167+
decodes strings.
1168+
1169+
If *translate* is set to true, translate ``'\r\n'`` and ``'\r'`` into
1170+
``'\n'``.
1171+
1172+
The *errors* argument works like that of
1173+
:class:`codecs.IncrementalDecoder`.
1174+
1175+
.. attribute:: newlines
1176+
1177+
a tuple of types of newlines encountered, or the newline if only
1178+
one type of newline is found. The possible values are
1179+
``None``, ``'\n'``, ``'\r'``, ``('\r', '\n')``, ``'\r\n'``,
1180+
``('\n', '\r\n')``, ``('\r', '\r\n')`` and
1181+
``('\r', '\n', '\r\n')``.
1182+
1183+
1184+
Example usage::
11501185

1186+
>>> import codecs
1187+
>>> import io
1188+
>>> idec = codecs.getincrementaldecoder("utf-8")()
1189+
>>> inld = io.IncrementalNewlineDecoder(idec, translate=True)
1190+
>>> inld.decode(b'1\r')
1191+
'1'
1192+
>>> inld.decode(b'\n2\n')
1193+
'\n2\n'
1194+
>>> inld.decode(b'3\r', final=True)
1195+
'3\n'
11511196

11521197
Static Typing
11531198
-------------

0 commit comments

Comments
 (0)