Add ability to generate Markdown "todo" list for GitHub/GitLab issues, Kanban boards, etc.

[david-a-wheeler]

We want people to merge "get a badge" into their wider workflows.

Add ability to generate Markdown "todo" list for GitHub/GitLab issues, Kanban boards, etc. Then the badge requirements can be easily merged into larger workflows.

[david-a-wheeler]

Maybe generate CSV for what's to do as well.

[lucasgonze]

Requirements for the project:

• There should be roughly equivalent functionality for every feature in the web app. We should attempt to be comprehensive. Omissions should be unavoidable.
• The state of the web app should be reflected in the markdown. If an item is Met in the web app it should be checked off in the Markdown.
• There should not be a feature to import from markdown back to the web app. There is no practical way.

Documentation on markdown for task lists:
https://github.github.com/gfm/#task-list-items-extension-

A useful feature is that task lists can be nested.

• The "show details" documentation should be made available in the markdown as a tooltip (because markdown allows HTML).

Markdown:
The project MUST provide basic documentation for the software produced by the project. ([documentation_basics](/uri "This documentation must be in some media (such as text or video) that includes: how to install it, how to start it, how to use it (possibly with a tutorial using examples), and how to use it securely (e.g., what to do and what not to do) if that is an appropriate topic for the software. The security documentation need not be long. The project MAY use hypertext links to non-project material as documentation. If the project does not produce software, choose 'not applicable' (N/A)."))

Rendered:

• Panels in the web app can be represented as nested task lists

HTML:

Markdown:

- [ ] Documentation
  - [ ]  The project MUST provide basic documentation for the software produced by the project. (documentation_basics) 
  - [ ]   The project MUST provide reference documentation that describes the external interface (both input and output) of the software produced by the project. (documentation_interface)

Rendered:

Badge criteria may be in three states: not yet researched, met, and not met.

We will represent those states with the following markdown:

 - [?] indeterminate 
 - [ ] not met
 - [x] met

The appearance of those is as follows:

The inconsistent appearance of the indeterminate state is unfortunate! You can’t make a checkbox indeterminate through HTML.

[SecurityCRob]

We discussed today in the BEST WG. The group was positive on the suggestion. David W. was going to explore more. Thanks for sharing!

[david-a-wheeler]

GitLab Flavored Markdown is defined at https://docs.gitlab.com/ee/user/markdown.html and it appears to include GitHub Flavored Markdown, which is great.

GitHub AND GitLab support:

• Not done
• Done

GitLab also supports this, but GitHub does not, so I think we should not use it:

• [~] Not applicable

[david-a-wheeler]

It looks like the simplest option is to map "Met" and "N/A" to [x] and map "Unmet" and "?" to [ ].

[lucasgonze]

Notes from BEST WG Jan 30, 2024

Requirements

• Export must be supported by github

Verify:

• Compatibility with Pandoc, Commonmark, Gitlab, and Jira

Documentation:

• Gitlab flavored markdown https://docs.gitlab.com/ee/user/markdown.html
• HTML5 "details" feature

Engineering:

• ? can be mapped to "Not met" state
• N/A can be mapped to "Met" state (Gitlab has tilde: - [~] Inapplicable task)
• The Best Practices web app already supports an extension on the URL. Append .json or .csv to set export format. As the entry point to this exporter, consider .md extension.

[david-a-wheeler]

It's not complete, but here's a partial start: #2104

[david-a-wheeler]

We now have a "secret" capability to generate todo lists in markdown. Just add ".md" to a project entry. It will return the project badge entry in markdown format:

• https://www.bestpractices.dev/en/projects/1.md
• https://www.bestpractices.dev/en/projects/1.md?criteria_level=1 for silver
• https://www.bestpractices.dev/en/projects/1.md?criteria_level=2 for gold

We can add links to this, but first we need to see if it's worth using :-). Thoughts?

[SecurityCRob]

I like it Cheers, CRob Director of Security Communications Intel Product Assurance and Security Book time with Robinson, Christopher ***@***.***?anonymous&ep=pcard> From: David A. Wheeler ***@***.***> Sent: Wednesday, January 31, 2024 11:31 AM To: coreinfrastructure/best-practices-badge ***@***.***> Cc: Robinson, Christopher ***@***.***>; Comment ***@***.***> Subject: Re: [coreinfrastructure/best-practices-badge] Add ability to generate Markdown "todo" list for GitHub/GitLab issues, Kanban boards, etc. (Issue #2094) We now have a "secret" capability to generate todo lists in markdown. Just add ".md" to a project entry. It will return the project badge entry in markdown format: * https://www.bestpractices.dev/en/projects/1.md * https://www.bestpractices.dev/en/projects/1.md?criteria_level=1 for silver * https://www.bestpractices.dev/en/projects/1.md?criteria_level=2 for gold We can add links to this, but first we need to see if it's worth using :-). Thoughts? — Reply to this email directly, view it on GitHub<#2094 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQRFDLDJ6CXQMBRDJPKHOO3YRJWVDAVCNFSM6AAAAABBRUXDX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJZGQ3DOMRZGA>. You are receiving this because you commented.Message ID: ***@***.******@***.***>>

[david-a-wheeler]

Note: This works in any supported language. E.g., for French instead of English use:

https://www.bestpractices.dev/fr/projects/1.md

[TonyLHansen]

It looks potentially useful.

[david-a-wheeler]

It looks potentially useful.

Great! That was certainly the goal :-).

Please try it out to confirm or deny the "it's useful" hypothesis. If someone confirms it is useful, we can add a link somewhere so people can find this new capability.

I intentionally generated a todo list for only one level. The todo list is already long for any one level; showing all levels at the same time would be overwhelming I think. The goal is to help people complete the work, not to overwhelm them :-).

[lucasgonze]

The todo list is already long for any one level; showing all levels at the same time would be overwhelming I think.

I tried out the output in a markdown editor (https://markdown-editor.github.io/) that can show just the rendered version. This was not too much. The code version is super hard to read, though.

I'm not sure it's tenable to show just one level. Fortunately the current output that I'm seeing does have all the levels.

The reason showing all the levels at the same time is overwhelming is that the best practices badge is substantive. There's a lot going on, but it's for good reason. It's just that projects working through it need to be organized and have follow-through.

[TonyLHansen]

Hmmm, can criteria_level be a list? as in "?criteria_level=1,2,3". That could let you get just the single level, or multiple levels as needed.

[david-a-wheeler]

It could be a list, but that's inconsistent with other things, and a pain to process.

An alternative would be to generate the full sequence if no level given, and only those criteria at a level if the level is specified. If you want more than one, you probably want them all :-).

[TonyLHansen]

+1

[david-a-wheeler]

The generated markdown seems to work really well at https://markdown-editor.github.io/ so that's encouraging.

[david-a-wheeler]

"No levels means all levels" drafted here:
#2106

[david-a-wheeler]

You can now get todos in markdown.

To generate a markdown for all levels for project 1 in English:

• https://www.bestpractices.dev/en/projects/1.md

You can request specific levels and/or a different locale:

• https://www.bestpractices.dev/en/projects/1.md?criteria_level=1 for passing
• https://www.bestpractices.dev/en/projects/1.md?criteria_level=1 for silver
• https://www.bestpractices.dev/en/projects/1.md?criteria_level=2 for gold
• https://www.bestpractices.dev/fr/projects/1.md for all levels in French (etc.)

Checkboxes are filled in for "Met" and "N/A". They are empty for "Unmet" or "?" (Unknown).

What do you think?

[david-a-wheeler]

If this works, we now need to figure out how to let people find this capability. I suggest adding some text somewhere on a badge entry, e.g., when viewing https://www.bestpractices.dev/en/projects/1. Where should that text go? What should it say? What part would be a hyperlink to it? Suggestions welcome.

[david-a-wheeler]

Note: GitHub and GitLab support checkboxes. CommonMark doesn't support checkboxes. However, the only alternative is to use HTML, but many systems won't allow "raw" HTML & that stuff is hard to read. I think anyone who wants to use a checkbox-based todo is going to use a markdown system that supports checkboxes.

[david-a-wheeler]

(Reposting, there was a weird problem)

Pandoc also works well. I think support from GitHub, GitLab, and pandoc is sufficient. Here's the proof. Given file x.md:

- [x] This is <details><summary>[foo]</summary> some details</details>

And running command:

pandoc -f 'gfm+task_lists' -t html x.md

It produces:

<ul class="task-list">
<li><label><input type="checkbox" checked="" />This is
<details><summary>[foo]</summary> some details</details></label></li>
</ul>

I suggest noting the pandoc command in some "details" for the link to the markdown generated file.

[lucasgonze]

I think support from GitHub, GitLab, and pandoc is sufficient.

I agree. We're pushing the limits and it's reasonable to say that some things are not possible.

[lucasgonze]

we now need to figure out how to let people find this capability.

Research

Is there already a way to find the JSON and (I think) XML export formats? I don't think so, but if there is then we should add one more format.

Glyphicons has these relevant icons:
Cloud download:
Save / Save As

This application has a multistep flow for exporting data that uses a modal dialog with a format chooser. It is great from a usability POV but too complicated as far as implementation:
https://www.frontitude.com/guides/export-project-content-as-json

Experiments

Experiment 1: on the panel title

That textual approach doesn't work at all. Possibly a Markdown icon that looks like a button could work.

Experiment 2: text in the footer

Add this:

<em class="glyphicon glyphicon-save"></em> Export data as <a href="">Markdown</a>, <a href="" class="glyphicon glyphicon-save">JSON</a>, <a href="">XML</a><br><br>

Just above:

This data is available under the Creative Commons Attribution version 3.0 or later license (CC-BY-3.0+).

Screenshot:

[david-a-wheeler]

Text at bottom makes sense. I'd want to be able to generate Markdown for two cases, "current level" and "all levels".

[lucasgonze]

Text at bottom makes sense. I'd want to be able to generate Markdown for two cases, "current level" and "all levels".

Mockup of current level:

Code:

This code displays an icon but is not itself an anchor or clickable target.

Note that the title attribute is good for accessibility and also creates a tooltip to explain the feature: