T Strings - Why there is no built in string rendering?
Posted by UsernamesArentClever@reddit | Python | View on Reddit | 59 comments
I like the idea of T Strings and here is a toy example:
cheese = "Brie"
template = t'We have {cheese} in stock.'
print(template.strings)
# → ('We have ', ' in stock.')
print(template.interpolations)
# → something like (Interpolation('Brie', 'cheese', None, ''),)
print(template.values)
# → ('Brie',)
But why isn't there a
print(template.render)
# → We have Brie in stock.
gerardwx@reddit
Here’s a package with some examples: https://pypi.org/project/tstring-util/
cbarrick@reddit
If you want a "default" rendering, use an f-string.
The point of t-strings is for custom renderers.
E.g. a SQL renderer will escape the interpolations to avoid SQL injection, and an HTML renderer will escape the interpolations to avoid HTML injection. They need separate renderers because the escape syntax is different between SQL and HTML.
Leseratte10@reddit
It would be cool to have an f-string with delayed rendering, though.
Behaves like an f-string as in there's no templating, escaping, and so on, but the variables are only replaced at time of render, not when the f-string is defined.
gerardwx@reddit
https://pypi.org/project/tstring-util/
R3D3-1@reddit
What you describe could be done with a lambda and an f-string.
The syntax isn't great, especially if you also want caching on first evaluation, but it is possible.
I generally keep running into situations, where I wish that Python had lazy-evaluating values, i.e. the equivalent of the cached functions above with arbitrary expressions, but with the evaluation being implicit when the value is actually used for something.
It would probably be possible to write a proxy class with that behavior, but typing support would be a big issue, and I'm not sure it is possible to get
type(deferred_obj)
return the type of the result or — better — a deferred value itself.E.g.
would need to evaluate the deferred expression, as does
deferred + 1
unless explicitly deferred itself, butfunc(deferred)
orlist0.append(deferred)
wouldn't need to evaluate it.But with the available syntax it would again require something verbose like
SheriffRoscoe@reddit
Indeed, all those points are covered in the PEP, as reasons why the authors chose to make t-strings use values, not lambdas.
firemark_pl@reddit
Maybe just use str.format like in old days.
Schmittfried@reddit
Not nearly as ergonomic for logging (and still not delayed unless you add level guards.
radicalbiscuit@reddit
Unsolicited advice: having custom info in the logging messages is an anti-pattern. Don't put variable info in the string; that's what the
extra
param is for. Good way to start structured logging, too!Schmittfried@reddit
Unsolicited disagreement: I want some info in both. The message is what I see first and often doesn’t make much sense without some dynamic parts.
radicalbiscuit@reddit
Yeah I'm not recommending absolute avoidance of dynamic messages, but I'm suggesting that, if you're using f strings in most logging statements, you might be engaging in an anti-pattern.
Maybe I'd refine my suggestion to say you want the message to be specific enough to fit the error/incident/scenario in the broadest way possible that still gives a you forensic edge. You don't want messages to be as broad as, "Something happened," but you also don't want it to be, "A murder happened to Col. Mustard at 4:39pm in the conservatory with a candlestick." You want it to be specific enough that you can search for all occurrences in the same bucket, but if you're making it dynamic when a static string with params will do, it's at least mildly harmful.
Would you agree with that nuance?
gmes78@reddit
Only because logging libraries don't support it.
TSM-@reddit
Yeah thats how you do it.
The
f
prefix is for when you want it rendered immediately.No_Hovercraft_2643@reddit
but fstrings allowed to have only a subset of the data, while format don't
njharman@reddit
f is also (and more importantly) rendered from context.
These are both rendered immediately. f-string is mostly syntactic sugar; reducing boilerplate and repetition by automatically interpolating from context.
firemark_pl@reddit
Dude I know :D
Penetal@reddit
I didn't
SheriffRoscoe@reddit
Of course, that wouldn’t be what t-strings are. t-string interpolations are eagerly evaluated where they’re defined, exactly like f-strings. The Template object stores the results of those evaluations.
billsil@reddit
Agreed. Thought that’s what a t string was until now.
ParentPostLacksWang@reddit
Escaping to make SQL queries is almost never the correct approach. DB engines support binding for a reason. It should be your first-line approach to getting variables into your SQL queries, and you should generally bend over backwards not to be relying on string escaping in any way.
rapture_survivor@reddit
From a language design perspective, what would be wrong with the print() function accepting a t-string and printing it out as plain text? It seems to me that if you're passing a t-string to print(), it is obvious the intent is to -not- interpolate.
This is coming from other languages where functionality like the t-string has been implemented transparently. In C# string interpolation values can be escaped by my DB library if they decide to accept the correct parameters at the call site. Otherwise, the interpolation behaves exactly like a string. To me this seems easy because it is super simple when writing application code.
Am I missing something? Is this a python specific problem?
SheriffRoscoe@reddit
True.
Nope. An SQL renderer will use the prepared-statement API to avoid interpolating the values into the statement in the first place. This is actually the perfect use of t-strings.
AngusMcBurger@reddit
All the MySQL clients I know of in Python (pymysql/mysqlclient/mysql-connector-python) escape parameters client-side, and substitute them directly into the query string like OP described. So if those clients add support for template strings, that's exactly how they'll do it
james_pic@reddit
That's more a limitation of MySQL's wire format than anything else. I know PostgreSQL and Oracle at very least support parametrization at the wire level, and these are implemented by their Python drivers.
cbarrick@reddit
True. Nothing about t-strings requires the renderer to output a string. You could very well output a request structure to send to your database.
Most of the examples in the PEP 750 are string oriented, so I assume that is still the primary use case.
madness_of_the_order@reddit
There are still reasons to render at least art of sql template. For example you may want to template table name or schema or column alias
ThiefMaster@reddit
Yes, and a proper SQL handler for a t-string would be smart enough to recognize the context where an interpolation is used, and either escape it (table/column name) or parametrize it (value).
AnythingApplied@reddit
From PEP 750 – Template Strings:
CelDaemon@reddit
https://docs.python.org/3/library/string.html#string.Template.substitute
CelDaemon@reddit
https://docs.python.org/3/library/string.html#string.Template.substitute It's already a thing, why is no one talking about this
Schmittfried@reddit
Contrary to what the others are saying, I do think a string renderer as part of the standard lib should be a thing. It should be a deliberate choice to avoid injection vulnerabilities caused by code that follows the path of least resistance, but it should be possible to render them as a plain old string without having to implement that yourself (esp. since the library version could be implemented in C).
First use case I can think of is custom logging libraries where you want to allow deferred interpolation. f-string is not the answer here and I‘m not convinced this is too small of a use case to warrant a standard implementation.
dandydev@reddit
You can do that. Use an f-string
cointoss3@reddit
Exactly. If you want to actually render a sanitized string, how is the stdlib supposed to know how to treat the variables? You need a thin wrapper to do that, and poof, there’s your renderer.
Schmittfried@reddit
I don’t need a sanitized string, I need deferred f-string interpolation, e.g. for logging.
SheriffRoscoe@reddit
Really? Everybody else I’ve ever seen make a similar comment actually wants deferred evaluation of the expressions, which t-strings don’t provide. The claim is always that deferring evaluation makes using “expensive” expressions more tolerable, in the case where the log message is discarded.
jmpjanny@reddit
t-strings are evaluated eagerly, not lazily. They are not the solution for deferred evaluation.
cointoss3@reddit
Then use template strings? They have existed since 2005.
gmes78@reddit
That's something that the logging library needs to do. No new string type is needed for that.
damesca@reddit
Then use template strings and write a template renderer.
Or use the standard deferred interpolation in the stdlib logging library.
Neither of these things are difficult now.
Schmittfried@reddit
Or you read my original comment again. Do you just not want to get it?
I specifically said:
and:
Your options are missing the entire point. Again.
Schmittfried@reddit
Read my comment again.
cointoss3@reddit
Because at that point, you basically have an f-string. This is not what template strings are for.
commy2@reddit
These are such an odd feature. Why are they named t-strings anyway? They are emphatically not strings, but templates. Shouldn't they named be t-templates? "String-templates" (instead of "Template-strings")?
cointoss3@reddit
Because to use one, you add t before the string…
It’s not an odd feature. It’s really great, you just don’t seem to understand them.
commy2@reddit
I certainly don't understand why they had to misname them like that.
VictoryMotel@reddit
Is string substitution called "rendering" now?
Quasar6@reddit
Because the intent is that your processing of the T string can produce any type, so it doesn’t make sense to have a default for str
secret_o_squirrel@reddit
Oh huh ok. I wondered this myself. I still think basically defaulting to “render-time-f-strings” instead of “assignment-time-f-strings” would be cool…but admittedly writing that renderer is very few lines of code.
snugar_i@reddit
Which people will have to write over and over again. I agree it should've been a part of stdlib
Quasar6@reddit
No they don't because if you want the result type to be a string then use an f-string. T-strings are a generalization of f-strings. So what you mention that it needs to be written "over and over" is false. The functionality is already there!
XtremeGoose@reddit
The difference between a t string and an f string (if a t string rendered to a string) is that the former are lazy and that is useful.
Why do you think people still use % templates for logging?
Jhuyt@reddit
Because I think one of the points of t-strings is that they are flexible in how they are rendered. An SQL request should be rendered differently from an HTML document, so having a default renderer does not make obvious sense to me
UsernamesArentClever@reddit (OP)
Ah, I don't understand why an SQL request should be rendered differently from an HTML document. I'll look into that.
Resquid@reddit
That is definitely a key idea here.
Jhuyt@reddit
It's because you need to escape certain characters for HTML to render correctly and SQL to be secure. In particular quotation marks are a struggle IIUC, but there are other things too.
james_pic@reddit
Note also that, for SQL, and potentially a few other use cases like GraphQL, it may not even make use of escaping when all is said and done, since these use cases support parameterisation, so queries and templated parameters can travel separately to the backend.
CrackerJackKittyCat@reddit
They have different escaping rules.
Read up on both HTML and SQL injection attacks.
JanEric1@reddit
Directly from the PEP: https://peps.python.org/pep-0750/#no-template-str-implementation
Gainside@reddit
I had the same “why not just render it?” thought until I realized that’s exactly what `f-strings` already do. T-strings are more like ASTs you can manipulate—you’re supposed to decide how to render them.