Clear user defined TeX macros from global state

[rianmcguire]

MathJax supports TeX macros, and once defined, they persist forever in the global state of MathJax-node.

This PR changes the behaviour so any definitions are cleared before typeset, but still allows them to be persisted across multiple typesets using the state option.

[rianmcguire]

Sorry - looks like I should have forked off develop, not master. Fixing now.

[dpvc]

Since you use TeX.Definitions.macros a number of times, here, perhaps

var MACROS = TeX.Definitions.macros;

would be in order.

(Also, there should be a space between the ) and the following {.)

[dpvc]

You should use if (state.macros.hasOwnProperty(macroName) {...} inside this loop, so that you will be sure to only use the names that are actually in the object. For example, if someone augments the Object.prototype object (and we have encountered that before), then you will loop through this additions as well.

[dpvc]

Should not need to set this, as it is already set in the objects that are pointed to in state.macros.

[rianmcguire]

I figured if someone were to serialize the state to JSON, the isUser property would be lost because the macro is an array.

If that's not something you want to support, I'm happy to remove it. It's not important for our use case at the moment.

[dpvc]

Again, if (TeX.Definitions.macros.hasOwnProperty(maroName)) {...} should be used inside this loop.

[dpvc]

I like the idea of this, but I've made some review comments above. A couple of additional thoughts:

• You might consider adding user-defined environments to the state as well (those created via \newenvironment).

• You should note that this does not remove macros loaded from extensions, either autoloader or explicitly loaded. So someone typesetting \require{color} {\color{red} x}y followed by \color{red}{x} y would get a red y in the second one. I don't think there is anything to be done about that, but just wanted to point out that this is not a total reset.

• The loop through all the defined macros (to remove user-defined ones) is being done every time an expression is typeset. That seems a bit inefficient. Perhaps you could record the number of macros before the typesetting occurs, and only do the loop if that number changes. Probably not a big deal, but every bit helps.

[dpvc]

In terms of my last bullet point above, there are methods setDef and setEnv that are used to create any user-defined macro and environment definitions. These could be overridden to record the newly created macros and environments, so you know exactly which ones have been created, rather than having to do any looping through the definitions at all. These are used by the begingroup extension, but could be used here, too, with a little care. Just a thought.

[rianmcguire]

Thanks for the feedback. I liked your setDef/setEnv idea, so I've re-implemented it by adding overrides for those.

[dpvc]

I appreciate your continued work on this, and like the direction this is going. I do have two issues that still concern me a bit.

First, if the begingroup extension is loaded (say via \require{begingroup}), then your setDef and setEnv will be overridden. So you probably should do something similar to what you have done for newcommand in order to tie into them again (I think a direct copy of what you have but with the extension name changed would do it).

Second, if someone redefines an existing macro (e.g., \def\phi{\varphi}), then when you remove the user-defined macros, there will be no definition for \phi any more. So you might want your setDef and setEnv replacements to be a little more sophisticated, and check if there is a (non-user-defined) version of the macro already, and save it for restoration when the user-defined macros are removed.

There is still one hitch even with that: extensions that define macros will not override user-defined macros (i.e., they act as though the extension had been loaded initially, even if they are auto-loaded later). That means that if typesetting causes macro to be defined with the same name as one in an extension and then loads the extension, you will remove the user-defined one, but not restore the extension-loaded version that should be there in the absence of user-defined macros.

This behavior is controlled by the TeX.Definitions.Add() method. You could override that to check if a definition is not being applied due to a user-defined one already existing, and save the new definition for restoration later.

Of course, this is a pretty low-likelihood situation, but it is one of those places where it could cause confusion about macros not being defined that should be.

I suppose an alternative would be to just end up using the begingroup extension all the time, as it properly handles user-supplied macros, and add a "reset" method that simply clears all the user data and local definition groups. Someone proposed such a reset (as a macro) at one point — I may be able to dig that up somewhere.

[pkra]

@rianmcguire any chance you'll get back to this some time soon? It would make a nice addition to 1.2.0 but it can easily wait for a later release.

[rianmcguire]

@pkra I'm not going to have time to get back to it for at least 2 weeks.

[pkra]

@rianmcguire thanks for the update!

[pkra]

@rianmcguire ping.

[Menci]

So, any progress here?

[dpvc]

So, any progress here?

I've been thinking about a different approach that uses the begingroup extension and a custom reset macro that will remove any user-defined macros (including those made with \global or \gdef) when the reset command is given. So you can process the math on a page without resetting, and have the definitions affect later expressions, and then reset if you want to start a new page fresh.

Here's how to do it right now. Make a directory named extensions and create the file extensions/reset.js containing

MathJax.Hub.Register.StartupHook("TeX begingroup Ready", function () {
  var TEX = MathJax.InputJax.TeX, NSSTACK = TEX.nsStack, NSFRAME = NSSTACK.nsFrame;
  var rootStack = TEX.rootStack, FIND = rootStack.Find.bind(rootStack);
  rootStack.globalFrame = rootStack.stack[0];      // save the original global definitions
  rootStack.stack[0] = NSFRAME();                  // a new (temporary) global definition holder
  rootStack.Find = function (name, type) {
    if (name === 'ReSeT' && type === "macros") return "ReSeT";       // don't let this be redefined
    return FIND(name, type) || this.globalFrame.Find(name, type);    // look up the name
  }
  TEX.Definitions.macros.ReSeT = 'ReSeT';        // define the reset macro to clear the stack
  TEX.Parse.Augment({ReSeT: function (name) {rootStack.Clear()}});
});

MathJax.Ajax.loadComplete("[extensions]/reset.js");

Then in your application for MathJax, configure the path to the extensions directory, load [extensions]/reset.js, TeX/begingroup.js, and TeX/newcommand.js, and process \ReSeT whenever you want to reset the macros to their initial state.

For example, this driver file (tex-reset.js)

#! /usr/bin/env node

const path = require('path');
const mjAPI = require("./lib/main.js");

mjAPI.config({
  extensions: '[extensions]/reset.js,TeX/begingroup.js,TeX/newcommand.js',
  paths: {extensions: path.join(__dirname,'extensions/')}
});
mjAPI.start();

mjAPI.typeset({
  math: process.argv[2],
  format: "inline-TeX",
  mml:true
}, (data) => !data.errors && console.log(data.mml));

mjAPI.typeset({
  math: '\\ReSeT',
  format: 'inline-TeX'
}, () => {});

mjAPI.typeset({
  math: process.argv[3],
  format: "inline-TeX",
  mml:true
}, (data) => !data.errors && console.log(data.mml));

takes two arguments (TeX expressions), and processes them with a \ReSeT in between. So you can test if this works by, for example,

node tex-reset '\def\sin{\cos} \sin(x)' '\sin(x)'

The output of the first should be a cosine, and the second a sine. You can also check that \ReSeT can't be redefined

node tex-reset '\def\ReSeT{Boom!} \def\sin{\cos} \sin(x)' '\sin(x)'

This is based on an idea of Imari Karonen on the Math StackExchange site.

This does not make the definitions available in the data object, as this PR does, but I'm not sure that is necessary. Also, neither this alternative nor the original PR prevents macros defined by loading extensions, so if an expression uses \require{color}, for example, then the \color macro will remain the LaTeX-compatible one rather than the non-standard MathJax-specific one.

To avoid this, you could disable the \require command, or use the Safe extension to limit the extensions that are allowed to be loaded (to disallow color). Or you could track the use of \require{} and when you want to reset the definitions, if \require{} has been used you could restart MathJax via mjAPI.start() rather than typesetting \ReSeT. That is more expensive, of course, but it is the only way to remove the fact that an extension has been loaded.

[dpvc]

The last commit should not be included, here. As mentioned in #451, the change is not needed. Also, it is unrelated to the changes for TeX macros that are the subject of this PR, so should not be part of this PR. Had #451 been merged, there would be no need for this commit anyway.

One way to roll it back would be:

git checkout master
git reset --hard HEAD~1
git push --force origin master

If you are using this branch for your production work, and have included it for that, then you should not be using that branch for a pull request (it is best to make a separate branch for the PR, rather than using your own master branch). Because there is already a lot of important comments on this PR with this branch, it would be hard to change it now. But you could branch a new "production" branch from your master branch and use that for your additional commits like this one.