-
Notifications
You must be signed in to change notification settings - Fork 28.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-46437][DOCS] Add custom tags for conditional Jekyll includes
### What changes were proposed in this pull request? Add [custom Jekyll tags][custom] to enable us to conditionally include files in our documentation build in a more user-friendly manner. [This example][example] demonstrates how a custom tag can build on one of Jekyll's built-in tags. [custom]: https://github.com/Shopify/liquid/wiki/Liquid-for-Programmers#create-your-own-tags [example]: Shopify/liquid#370 (comment) Without this change, files have to be included as follows: ```liquid {% for static_file in site.static_files %} {% if static_file.name == 'generated-agg-funcs-table.html' %} {% include_relative generated-agg-funcs-table.html %} {% break %} {% endif %} {% endfor %} ``` With this change, they can be included more intuitively in one of two ways: ```liquid {% include_relative_if_exists generated-agg-funcs-table.html %} {% include_api_gen generated-agg-funcs-table.html %} ``` `include_relative_if_exists` includes a file if it exists and substitutes an HTML comment if not. Use this tag when it's always OK for an include not to exist. `include_api_gen` includes a file if it exists. If it doesn't, it tolerates the missing file only if one of the `SKIP_` flags is set. Otherwise it raises an error. Use this tag for includes that are generated for the language APIs. These files are required to generate complete documentation, but we tolerate their absence during development---i.e. when a skip flag is set. `include_api_gen` will place a visible text placeholder in the document and post a warning to the console to indicate that missing API files are being tolerated. ```sh $ SKIP_API=1 bundle exec jekyll build Configuration file: /Users/nchammas/dev/nchammas/spark/docs/_config.yml Source: /Users/nchammas/dev/nchammas/spark/docs Destination: /Users/nchammas/dev/nchammas/spark/docs/_site Incremental build: disabled. Enable with --incremental Generating... Warning: Tolerating missing API files because the following skip flags are set: SKIP_API done in 1.703 seconds. Auto-regeneration: disabled. Use --watch to enable. ``` This PR supersedes #44393. ### Why are the changes needed? Jekyll does not have a succinct way to [check if a file exists][check], so the required directives to implement such functionality are very cumbersome. We need the ability to do this so that we can [build the docs successfully with `SKIP_API=1`][build], since many includes reference files that are only generated when `SKIP_API` is _not_ set. [check]: jekyll/jekyll#7528 [build]: #44627 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually building and reviewing the docs, both with and without `SKIP_API=1`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #44630 from nchammas/SPARK-46437-conditional-jekyll-include. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
- Loading branch information
1 parent
d203328
commit 6679419
Showing
3 changed files
with
96 additions
and
141 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| module Jekyll | ||
| # Tag for including a file if it exists. | ||
| class IncludeRelativeIfExistsTag < Tags::IncludeRelativeTag | ||
| def render(context) | ||
| super | ||
| rescue IOError | ||
| "<!-- missing include: #{@file} -->" | ||
| end | ||
| end | ||
|
|
||
| # Tag for including files generated as part of the various language APIs. | ||
| # If a SKIP_ flag is set, tolerate missing files. If not, raise an error. | ||
| class IncludeApiGenTag < Tags::IncludeRelativeTag | ||
| @@displayed_warning = false | ||
|
|
||
| def render(context) | ||
| super | ||
| rescue IOError => e | ||
| skip_flags = [ | ||
| 'SKIP_API', | ||
| 'SKIP_SCALADOC', | ||
| 'SKIP_PYTHONDOC', | ||
| 'SKIP_RDOC', | ||
| 'SKIP_SQLDOC', | ||
| ] | ||
| set_flags = skip_flags.select { |flag| ENV[flag] } | ||
| # A more sophisticated approach would be to accept a tag parameter | ||
| # specifying the relevant API so we tolerate missing files only for | ||
| # APIs that are explicitly skipped. But this is unnecessary for now. | ||
| # Instead, we simply tolerate missing files if _any_ skip flag is set. | ||
| if set_flags.any? then | ||
| set_flags_string = set_flags.join(', ') | ||
| if !@@displayed_warning then | ||
| STDERR.puts "Warning: Tolerating missing API files because the " \ | ||
| "following skip flags are set: #{set_flags_string}" | ||
| @@displayed_warning = true | ||
| end | ||
| # "skip flags set: `#{set_flags_string}`; " \ | ||
| "placeholder for missing API include: `#{@file}`" | ||
| else | ||
| raise e | ||
| end | ||
| end | ||
| end | ||
| end | ||
|
|
||
| Liquid::Template.register_tag( | ||
| 'include_relative_if_exists', | ||
| Jekyll::IncludeRelativeIfExistsTag, | ||
| ) | ||
|
|
||
| Liquid::Template.register_tag( | ||
| 'include_api_gen', | ||
| Jekyll::IncludeApiGenTag, | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters