I used to work with a guy who put some examples in the README.md of a library repo, but none of them worked. When I asked him about it, he said, "They're examples. They aren't supposed to work."
I worked in a company where the examples would be run as tests as part of the unit test phase so if they didn't work then the deployment would fail. The best examples are the ones you can run
Examples are great at showing how the various pieces of the API come together to accomplish a specific task, and that's invaluable.
BUT examples are NOT a good place to discuss the subtleties and/or alternatives of each piece of the API, they absolutely do not show the pre-conditions and post-conditions, etc...
Yeah, I'm just pointing out that it is a tooling problem. Python documentation has tons of issues (why do none of the documentation systems have a "jump to source" button?), this is just one of them.
What? Unless I misunderstand you, why would you test use cases of libraries that your app... doesn't use?
The library's source itself should have tests ensuring proper function of optional parameters/edge cases. It is simply overkill and bad practice to write tests for the implementation of libraries that you are using, in the code base you use them. If you are that skeptical of a library, why bring it in at all?
I think the confusion here is that you are saying that tests are documentation thinking about your own code (which I agree with) but the other person is thinking about documentation for external users (like documentation for a library, where the people reading the docs largely aren't maintainers looking at the tests).
examples are NOT a good place to discuss the subtleties and/or alternatives of each piece of the API, they absolutely do not show the pre-conditions and post-conditions, etc...
The whole point of using examples to specify or document code is to clearly show how preconditions (given), the actions (when) and post conditions (then) relate to each other.
Pre-conditions and post-conditions are best documented (and asserted!).
For an example to document the pre-conditions, one would need to differentiate the incidental from the essential in the example.
If a function is invoked with 2 is it because an even number is required? A number between 0 and 10? Or is just that the author of the example picked off a random number and there's no precondition at all?
You can't tell from the example code, you can't tell at all.
The only good examples are code generated examples. If the code isn't keeping them up to date, you might as well throw them away and just read the code.
Even then, reading the actual code is the only real way to understand the intent of a codebase.
It's especially terrible in JavaScript where it's really common to have functions with 8 different overloaded argument signatures, and the examples show you 3 of them and they expect you to play around with it to figure out how they all work instead of just telling you in the docs. Like one of the parameters is called date and they show examples where it's an iso date string, and another where it's some custom date object the library created, but you see other places in the docs where it's a Unix timestamp as an integer and so you have no idea what the actual boundaries of this are. Does it take Temporal dates? What about other string date formats? Or the native Date object? Since you found other examples than are covered by the specific function's examples, clearly those examples aren't exhaustive and you can't tell if any other values that work are actually supported or if they only work by coincidence and might start failing in the future.
Or the function takes a callback function as an argument, and the examples show the callback function being passed a variable number of arguments, and one of them is some kind of object and it may be in the 2nd, 3rd, or 4th position and you can tell from the examples that it has at least 2 fields but from context you can assume it probably has more but they don't tell you so again have to experiment and hope that what you figure out from experimenting isn't liable to change without warning. And they show an example of the callback function being an async function but they don't make it clear if the function passed in is being awaited or not. And the function takes an argument called err but they only show checking if it's truthy and throwing it but they don't give you any idea of what the argument actually is.
Or one of the arguments is a string, and it's clear from context that there should be limits on what strings can be passed in (length, allowed characters, etc) but from examples you can't tell exactly what those limits are.
Or it's a string argument that is clearly some sort of enum, and they give examples that are all lowercase, and some that are camelCase, but you can't tell from that if casing doesn't matter or if there are just particular casings that are allowed. I know of at least one library that would allow some casings but not others and that was never explained in the docs, and others that convert everything to lowercase first so it doesn't matter which you use.
Most of this stuff would be easy to explain in a couple of sentences or with a type schema; examples are helpful but not sufficient on their own.
this is such a PM perspective but examples vs comprehensive docs is a user experience design choice, not a technical one.
examples are great for the 80% case - developers who just need to get something working quickly. they want to copy-paste and modify. comprehensive API docs are essential for the 20% case - edge cases, error handling, performance considerations.
the best doc sites understand this and structure around user intent: "getting started" sections with examples, then "reference" sections with exhaustive details. stripe's API docs are masterclass here - examples get you running in 5 minutes, but when you need to handle webhooks failing or rate limiting, the full docs are right there.
what frustrates me most is libraries that only do examples but don't tell you the failure modes. great for demos, terrible for production.
Strong disagree. Nothing is more annoying than documentation by example. Python libraries are the worst offenders, in my experience. Often the only recourse is to wade through the source code to figure out what the heck is going on.
JavaScript used to be really bad for this, but it seems to be getting a lot better now that typescript type definitions are basically expected for libraries. Like, if you are already managing type definitions it's not really a lot of extra work to output those into docs, so more people do it now.
This person is just discovering for themselves that Python documentation as a whole is just bad. Docs is something that the perl community did far better, and the standard in perl is to put sample usage at the BEGINNING of every manpage. I document all my python stuff that way as well, and encourage everybody to do the same.
I have a website that's all C++ examples but I think you need to be an experienced C++ programmer to benefit. I never meant for it to be a replacement for learncpp.com or cppreference.com or any other documentation.
I recently did a project using SDL2 and dear imgui, two C libraries. It took me about three times as long to figure out how to do do anything in imgui because the only documentation it has is an example app that showcases all the features. Learning SDL2 and figuring out how to use it was so much easier because of the docs
However i do think SDL2 docs would be better if they had examples along with everything else, kind of like cppreference
Well, yeah. The same way it depends on how good the written documentation is. Or how logically the API is written. Or if your monitor brightness is high enough to read the documentation.
ElderPimpx@reddit
The best documentation contains examples, but the examples themselves are not sufficient to make the best documentation.
iSpaYco@reddit
yep, you can't shove every scenario in examples without making unreadable,
however I do want to know everything I can pass to a function/api endpoint/etc.. otherwise I'm reading the codebase to learn about the program.
wokan@reddit
I used to work with a guy who put some examples in the README.md of a library repo, but none of them worked. When I asked him about it, he said, "They're examples. They aren't supposed to work."
I don't work with that dev anymore.
fagnerbrack@reddit (OP)
I worked in a company where the examples would be run as tests as part of the unit test phase so if they didn't work then the deployment would fail. The best examples are the ones you can run
nsjames1@reddit
I absolutely hate when the "docs" are just an examples directory in a GitHub repo.
It's the absolute worst.
matthieum@reddit
No, they're not.
Examples are great at showing how the various pieces of the API come together to accomplish a specific task, and that's invaluable.
BUT examples are NOT a good place to discuss the subtleties and/or alternatives of each piece of the API, they absolutely do not show the pre-conditions and post-conditions, etc...
aksdb@reddit
And good luck modeling all variations of optional parameters with examples.
polynomialcheesecake@reddit
And keeping it up to date
gmes78@reddit
In Rust, your examples also double as tests (AKA
doctests), so you'll just get an error if they're out of date.polynomialcheesecake@reddit
Yea that is pretty awesome I enjoy rust. But this article looks like python no?
gmes78@reddit
Yeah, I'm just pointing out that it is a tooling problem. Python documentation has tons of issues (why do none of the documentation systems have a "jump to source" button?), this is just one of them.
TexZK@reddit
Sphinx has "jump to source" AFAIK, and doctest is a thing. Adding examples to the docs is good manners.
gmes78@reddit
It's an extension that has to be explicitly enabled.
And, importantly, the standard library docs do not have this feature.
masklinn@reddit
You can do it in python as well: https://docs.pytest.org/en/stable/how-to/doctest.html
polynomialcheesecake@reddit
Nice
Ythio@reddit
Living documentation isn't a Rust specific concept.
gmes78@reddit
Didn't say it was.
araujoms@reddit
In Julia as well.
TheoreticalDumbass@reddit
one would hope your CI would build examples as well
dkarlovi@reddit
That's what your tests are. Your tests are the examples, are you saying you don't model "all variants of optional parameters" in tests?
tinieblast@reddit
What? Unless I misunderstand you, why would you test use cases of libraries that your app... doesn't use?
The library's source itself should have tests ensuring proper function of optional parameters/edge cases. It is simply overkill and bad practice to write tests for the implementation of libraries that you are using, in the code base you use them. If you are that skeptical of a library, why bring it in at all?
dkarlovi@reddit
What are you talking about?
gyroda@reddit
I think the confusion here is that you are saying that tests are documentation thinking about your own code (which I agree with) but the other person is thinking about documentation for external users (like documentation for a library, where the people reading the docs largely aren't maintainers looking at the tests).
nickcash@reddit
Tests are rarely public facing like an API doc
MoreRespectForQA@reddit
The whole point of using examples to specify or document code is to clearly show how preconditions (given), the actions (when) and post conditions (then) relate to each other.
matthieum@reddit
I disagree.
Pre-conditions and post-conditions are best documented (and asserted!).
For an example to document the pre-conditions, one would need to differentiate the incidental from the essential in the example.
If a function is invoked with
2is it because an even number is required? A number between 0 and 10? Or is just that the author of the example picked off a random number and there's no precondition at all?You can't tell from the example code, you can't tell at all.
corbymatt@reddit
The only good examples are code generated examples. If the code isn't keeping them up to date, you might as well throw them away and just read the code.
Even then, reading the actual code is the only real way to understand the intent of a codebase.
777777thats7sevens@reddit
It's especially terrible in JavaScript where it's really common to have functions with 8 different overloaded argument signatures, and the examples show you 3 of them and they expect you to play around with it to figure out how they all work instead of just telling you in the docs. Like one of the parameters is called
dateand they show examples where it's an iso date string, and another where it's some custom date object the library created, but you see other places in the docs where it's a Unix timestamp as an integer and so you have no idea what the actual boundaries of this are. Does it take Temporal dates? What about other string date formats? Or the native Date object? Since you found other examples than are covered by the specific function's examples, clearly those examples aren't exhaustive and you can't tell if any other values that work are actually supported or if they only work by coincidence and might start failing in the future.Or the function takes a callback function as an argument, and the examples show the callback function being passed a variable number of arguments, and one of them is some kind of object and it may be in the 2nd, 3rd, or 4th position and you can tell from the examples that it has at least 2 fields but from context you can assume it probably has more but they don't tell you so again have to experiment and hope that what you figure out from experimenting isn't liable to change without warning. And they show an example of the callback function being an async function but they don't make it clear if the function passed in is being awaited or not. And the function takes an argument called
errbut they only show checking if it's truthy and throwing it but they don't give you any idea of what the argument actually is.Or one of the arguments is a string, and it's clear from context that there should be limits on what strings can be passed in (length, allowed characters, etc) but from examples you can't tell exactly what those limits are.
Or it's a string argument that is clearly some sort of enum, and they give examples that are all lowercase, and some that are camelCase, but you can't tell from that if casing doesn't matter or if there are just particular casings that are allowed. I know of at least one library that would allow some casings but not others and that was never explained in the docs, and others that convert everything to lowercase first so it doesn't matter which you use.
Most of this stuff would be easy to explain in a couple of sentences or with a type schema; examples are helpful but not sufficient on their own.
Yawaworth001@reddit
There are still untyped JavaScript libraries in the wild?
Plank_With_A_Nail_In@reddit
Most API's aren't complex though lol. For the vast majority examples are good enough.
iamakramsalim@reddit
this is such a PM perspective but examples vs comprehensive docs is a user experience design choice, not a technical one.
examples are great for the 80% case - developers who just need to get something working quickly. they want to copy-paste and modify. comprehensive API docs are essential for the 20% case - edge cases, error handling, performance considerations.
the best doc sites understand this and structure around user intent: "getting started" sections with examples, then "reference" sections with exhaustive details. stripe's API docs are masterclass here - examples get you running in 5 minutes, but when you need to handle webhooks failing or rate limiting, the full docs are right there.
what frustrates me most is libraries that only do examples but don't tell you the failure modes. great for demos, terrible for production.
gyroda@reddit
Microsoft is terrible for this in the .Net ecosystem.
Great "getting started" guides, terrible at telling you what the different configuration options actually mean.
max123246@reddit
Yup, you need both to be useful
rehevkor5@reddit
Strong disagree. Nothing is more annoying than documentation by example. Python libraries are the worst offenders, in my experience. Often the only recourse is to wade through the source code to figure out what the heck is going on.
random_cornerme@reddit
Why is it annoying when an example is showing in addition to text documentation?
rehevkor5@reddit
I'm not annoyed by the presence of examples. I'm annoyed with documentation by example. The post claims, "examples are the best documentation".
backfire10z@reddit
Obviously you should not be using the code in any way other than how the example shows
777777thats7sevens@reddit
JavaScript used to be really bad for this, but it seems to be getting a lot better now that typescript type definitions are basically expected for libraries. Like, if you are already managing type definitions it's not really a lot of extra work to output those into docs, so more people do it now.
RiftHunter4@reddit
People just post stuff to this subreddit and 90% og the time its insane.
fagnerbrack@reddit (OP)
Can you elaborate?
dima55@reddit
This person is just discovering for themselves that Python documentation as a whole is just bad. Docs is something that the perl community did far better, and the standard in perl is to put sample usage at the BEGINNING of every manpage. I document all my python stuff that way as well, and encourage everybody to do the same.
ChemicalBrother812@reddit
I'd say examples are the simplest form of good documentation.
Great documentation also has comprehensive schema and usage information, not just examples.
UndeadMurky@reddit
Need both. I hate documentation that only shows examples, and I hate documentation that doesn't show examples.
CommodoreKrusty@reddit
I have a website that's all C++ examples but I think you need to be an experienced C++ programmer to benefit. I never meant for it to be a replacement for learncpp.com or cppreference.com or any other documentation.
sigmagoonsixtynine@reddit
Hard disagree
I recently did a project using SDL2 and dear imgui, two C libraries. It took me about three times as long to figure out how to do do anything in imgui because the only documentation it has is an example app that showcases all the features. Learning SDL2 and figuring out how to use it was so much easier because of the docs
However i do think SDL2 docs would be better if they had examples along with everything else, kind of like cppreference
ryxxel@reddit
In college, I was taught that unit tests are the best way to document code.
pacific_plywood@reddit
https://diataxis.fr
EarlMarshal@reddit
Depends on how good these example explore the surface of possible solutions with said APIs.
pepejovi@reddit
Well, yeah. The same way it depends on how good the written documentation is. Or how logically the API is written. Or if your monitor brightness is high enough to read the documentation.