Refactor inference optimizations

[xingniu]

This PR refactors inference optimizations:

• Make the inference_only mode switchable.
• Simplify inference optimizations by
  (1) using eval() to disable dropout instead of explicitly setting dropout modules to None;
  (2) always using default value inplace=False for activation modules.

Pull Request Checklist

• Changes are complete (if posting work-in-progress code, prefix your pull request title with '[WIP]'
  until you can check this box.
• Unit tests pass (pytest)
• Were system tests modified? If so did you run these at least 5 times to account for the variation across runs?
• System tests pass (pytest test/system)
• Passed code style checking (./style-check.sh)
• You have considered writing a test
• Updated major/minor version in sockeye/__init__.py. Major version bump if this is a backwards incompatible change.
• Updated CHANGELOG.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[mjdenkowski]

Thanks for working on this! I've added a few comments.

[mjdenkowski]

Thanks Xing! Can you make another pass to double check that any class that has a set_inference_only method also sets self.inference_only in both set_inference_only and __init__? The idea is that any class that supports switching in/out of inference mode can always be queried for its current state.

[mjdenkowski]

Looks good!