8274718: runtime/cds/appcds/LambdaEagerInit.java fails with -XX:-CompactStrings

Reviewed-by: iklam, ccheung

doc/building.html

@@ -488,7 +488,7 @@
 <li><code>CONF</code> and <code>CONF_NAME</code> - Selecting the configuration(s) to use. See <a href="#using-multiple-configurations">Using Multiple Configurations</a></li>
 </ul>
 <h4 id="test-make-control-variables">Test Make Control Variables</h4>
-<p>These make control variables only make sense when running tests. Please see <a href="testing.html">Testing the JDK</a> for details.</p>
+<p>These make control variables only make sense when running tests. Please see <strong>Testing the JDK</strong> (<a href="testing.html">html</a>, <a href="testing.md">markdown</a>) for details.</p>
 <ul>
 <li><code>TEST</code></li>
 <li><code>TEST_JOBS</code></li>
@@ -514,7 +514,7 @@
 </ul>
 <p>To execute the most basic tests (tier 1), use:</p>
 <pre><code>make run-test-tier1</code></pre>
-<p>For more details on how to run tests, please see the <a href="testing.html">Testing the JDK</a> document.</p>
+<p>For more details on how to run tests, please see <strong>Testing the JDK</strong> (<a href="testing.html">html</a>, <a href="testing.md">markdown</a>).</p>
 <h2 id="cross-compiling">Cross-compiling</h2>
 <p>Cross-compiling means using one platform (the <em>build</em> platform) to generate output that can ran on another platform (the <em>target</em> platform).</p>
 <p>The typical reason for cross-compiling is that the build is performed on a more powerful desktop computer, but the resulting binaries will be able to run on a different, typically low-performing system. Most of the complications that arise when building for embedded is due to this separation of <em>build</em> and <em>target</em> systems.</p>

doc/building.md

@@ -818,7 +818,7 @@ configuration, as opposed to the "configure time" configuration.
 #### Test Make Control Variables
 
 These make control variables only make sense when running tests. Please see
-[Testing the JDK](testing.html) for details.
+**Testing the JDK** ([html](testing.html), [markdown](testing.md)) for details.
 
   * `TEST`
   * `TEST_JOBS`
@@ -865,8 +865,8 @@ To execute the most basic tests (tier 1), use:
 make run-test-tier1
 ```
 
-For more details on how to run tests, please see the [Testing
-the JDK](testing.html) document.
+For more details on how to run tests, please see **Testing the JDK**
+([html](testing.html), [markdown](testing.md)).
 
 ## Cross-compiling
 

doc/hotspot-style.html

@@ -50,6 +50,7 @@
 <li><a href="#nullptr">nullptr</a></li>
 <li><a href="#atomic">&lt;atomic&gt;</a></li>
 <li><a href="#uniform-initialization">Uniform Initialization</a></li>
+<li><a href="#local-function-objects">Local Function Objects</a></li>
 <li><a href="#additional-permitted-features">Additional Permitted Features</a></li>
 <li><a href="#excluded-features">Excluded Features</a></li>
 <li><a href="#undecided-features">Undecided Features</a></li>
@@ -194,7 +195,7 @@ while ( test_foo(args...) ) { // No, excess spaces around control</code></pre></
 <p>Similar discussions for some other projects:</p>
 <ul>
 <li><p><a href="https://google.github.io/styleguide/cppguide.html">Google C++ Style Guide</a> — Currently (2020) targeting C++17.</p></li>
-<li><p><a href="https://chromium-cpp.appspot.com">C++11 and C++14 use in Chromium</a> — Categorizes features as allowed, banned, or to be discussed.</p></li>
+<li><p><a href="https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++11.md">C++11 and C++14 use in Chromium</a> — Categorizes features as allowed, banned, or to be discussed.</p></li>
 <li><p><a href="https://llvm.org/docs/CodingStandards.html">llvm Coding Standards</a> — Currently (2020) targeting C++14.</p></li>
 <li><p><a href="https://firefox-source-docs.mozilla.org/code-quality/coding-style/using_cxx_in_firefox_code.html">Using C++ in Mozilla code</a> — C++17 support is required for recent versions (2020).</p></li>
 </ul>
@@ -254,8 +255,7 @@ while ( test_foo(args...) ) { // No, excess spaces around control</code></pre></
 <li><p>Function argument deduction. This is always permitted, and indeed encouraged. It is nearly always better to allow the type of a function template argument to be deduced rather than explicitly specified.</p></li>
 <li><p><code>auto</code> variable declarations (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1984.pdf">n1984</a>)<br> For local variables, this can be used to make the code clearer by eliminating type information that is obvious or irrelevant. Excessive use can make code much harder to understand.</p></li>
 <li><p>Function return type deduction (<a href="https://isocpp.org/files/papers/N3638.html">n3638</a>)<br> Only use if the function body has a very small number of <code>return</code> statements, and generally relatively little other code.</p></li>
-<li><p>Generic lambdas. Lambdas are not (yet) permitted.</p></li>
-<li><p>Lambda init captures. Lambdas are not (yet) permitted.</p></li>
+<li><p>Also see <a href="#lambdaexpressions">lambda expressions</a>.</p></li>
 </ul>
 <h3 id="expression-sfinae">Expression SFINAE</h3>
 <p><a href="https://en.cppreference.com/w/cpp/language/sfinae" title="Substitution Failure Is Not An Error">Substitution Failure Is Not An Error</a> (SFINAE) is a template metaprogramming technique that makes use of template parameter substitution failures to make compile-time decisions.</p>
@@ -288,6 +288,121 @@ while ( test_foo(args...) ) { // No, excess spaces around control</code></pre></
 <li><a href="https://en.cppreference.com/w/cpp/language/aggregate_initialization">aggregate initialization</a></li>
 </ul>
 <p>Although related, the use of <code>std::initializer_list</code> remains forbidden, as part of the avoidance of the C++ Standard Library in HotSpot code.</p>
+<h3 id="local-function-objects">Local Function Objects</h3>
+<ul>
+<li>Local function objects, including lambda expressions, may be used.</li>
+<li>Lambda expressions must only be used as a downward value.</li>
+<li>Prefer <code>[&amp;]</code> as the capture list of a lambda expression.</li>
+<li>Return type deduction for lambda expressions is permitted, and indeed encouraged.</li>
+<li>An empty parameter list for a lambda expression may be elided.</li>
+<li>A lambda expression must not be <code>mutable</code>.</li>
+<li>Generic lambda expressions are permitted.</li>
+<li>Lambda expressions should be relatively simple.</li>
+<li>Anonymous lambda expressions should not overly clutter the enclosing expression.</li>
+<li>An anonymous lambda expression must not be directly invoked.</li>
+<li>Bind expressions are forbidden.</li>
+</ul>
+<p>Single-use function objects can be defined locally within a function, directly at the point of use. This is an alternative to having a function or function object class defined at class or namespace scope.</p>
+<p>This usage was somewhat limited by C++03, which does not permit such a class to be used as a template parameter. That restriction was removed by C++11 (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2657.htm">n2657</a>). Use of this feature is permitted.</p>
+<p>Many HotSpot protocols involve &quot;function-like&quot; objects that involve some named member function rather than a call operator. For example, a function that performs some action on all threads might be written as</p>
+<pre><code>void do_something() {
+  struct DoSomething : public ThreadClosure {
+    virtual void do_thread(Thread* t) {
+      ... do something with t ...
+    }
+  } closure;
+  Threads::threads_do(&amp;closure);
+}</code></pre>
+<p>HotSpot code has historically usually placed the DoSomething class at namespace (or sometimes class) scope. This separates the function's code from its use, often to the detriment of readability. It requires giving the class a globally unique name (if at namespace scope). It also loses the information that the class is intended for use in exactly one place, and does not have any subclasses. (However, the latter can now be indicated by declaring it <code>final</code>.) Often, for simplicity, a local class will skip things like access control and accessor functions, giving the enclosing function direct access to the implementation and eliminating some boilerplate that might be provided if the class is in some outer (more accessible) scope. On the other hand, if there is a lot of surrounding code in the function body or the local class is of significant size, defining it locally can increase clutter and reduce readability.</p>
+<p><a name="lambdaexpressions"></a> C++11 added <em>lambda expressions</em> as a new way to write a function object. Simple lambda expressions can be significantly more concise than a function object, eliminating a lot of boiler-plate. On the other hand, a complex lambda expression may not provide much, if any, readability benefit compared to an ordinary function object. Also, while a lambda can encapsulate a call to a &quot;function-like&quot; object, it cannot be used in place of such.</p>
+<p>A common use for local functions is as one-use <a href="https://en.cppreference.com/w/cpp/language/raii" title="Resource Acquisition Is Initialization">RAII</a> objects. The amount of boilerplate for a function object class (local or not) makes such usage somewhat clumsy and verbose. But with the help of a small amount of supporting utility code, lambdas work particularly well for this use case.</p>
+<p>Another use for local functions is <a href="https://en.wikipedia.org/wiki/Partial_application" title="Partial Application">partial application</a>. Again here, lambdas are typically much simpler and less verbose than function object classes.</p>
+<p>Because of these benefits, lambda expressions are permitted in HotSpot code, with some restrictions and usage guidance. An anonymous lambda is one which is passed directly as an argument. A named lambda is the value of a variable, which is its name.</p>
+<p>Lambda expressions should only be passed downward. In particular, a lambda should not be returned from a function or stored in a global variable, whether directly or as the value of a member of some other object. Lambda capture is syntactically subtle (by design), and propagating a lambda in such ways can easily pass references to captured values to places where they are no longer valid. In particular, members of the enclosing <code>this</code> object are effectively captured by reference, even if the default capture is by-value. For such uses-cases a function object class should be used to make the desired value capturing and propagation explicit.</p>
+<p>Limiting the capture list to <code>[&amp;]</code> (implicitly capture by reference) is a simplifying restriction that still provides good support for HotSpot usage, while reducing the cases a reader must recognize and understand.</p>
+<ul>
+<li><p>Many common lambda uses require reference capture. Not permitting it would substantially reduce the utility of lambdas.</p></li>
+<li><p>Referential transparency. Implicit reference capture makes variable references in the lambda body have the same meaning they would have in the enclosing code. There isn't a semantic barrier across which the meaning of a variable changes.</p></li>
+<li><p>Explicit reference capture introduces significant clutter, especially when lambda expressions are relatively small and simple, as they should be in HotSpot code.</p></li>
+<li><p>There are a number of reasons why by-value capture might be used, but for the most part they don't apply to HotSpot code, given other usage restrictions.</p>
+<ul>
+<li><p>A primary use-case for by-value capture is to support escaping uses, where values captured by-reference might become invalid. That use-case doesn't apply if only downward lambdas are used.</p></li>
+<li><p>By-value capture can also make a lambda-local copy for mutation, which requires making the lambda <code>mutable</code>; see below.</p></li>
+<li><p>By-value capture might be viewed as an optimization, avoiding any overhead for reference capture of cheap to copy values. But the compiler can often eliminate any such overhead.</p></li>
+<li><p>By-value capture by a non-<code>mutable</code> lambda makes the captured values const, preventing any modification by the lambda and making the captured value unaffected by modifications to the outer variable. But this only applies to captured auto variables, not member variables, and is inconsistent with referential transparency.</p></li>
+</ul></li>
+<li><p>Non-capturing lambdas (with an empty capture list - <code>[]</code>) have limited utility. There are cases where no captures are required (pure functions, for example), but if the function is small and simple then that's obvious anyway.</p></li>
+<li><p>Capture initializers (a C++14 feature - <a href="https://isocpp.org/files/papers/N3649.html">N3649</a>) are not permitted. Capture initializers inherently increase the complexity of the capture list, and provide little benefit over an additional in-scope local variable.</p></li>
+</ul>
+<p>The use of <code>mutable</code> lambda expressions is forbidden because there don't seem to be many, if any, good use-cases for them in HotSpot. A lambda expression needs to be mutable in order to modify a by-value captured value. But with only downward lambdas, such usage seems likely to be rare and complicated. It is better to use a function object class in any such cases that arise, rather than requiring all HotSpot developers to understand this relatively obscure feature.</p>
+<p>While it is possible to directly invoke an anonymous lambda expression, that feature should not be used, as such a form can be confusing to readers. Instead, name the lambda and call it by name.</p>
+<p>Some reasons to prefer a named lambda instead of an anonymous lambda are</p>
+<ul>
+<li><p>The body contains non-trivial control flow or declarations or other nested constructs.</p></li>
+<li><p>Its role in an argument list is hard to guess without examining the function declaration. Give it a name that indicates its purpose.</p></li>
+<li><p>It has an unusual capture list.</p></li>
+<li><p>It has a complex explicit return type or parameter types.</p></li>
+</ul>
+<p>Lambda expressions, and particularly anonymous lambda expressions, should be simple and compact. One-liners are good. Anonymous lambdas should usually be limited to a couple lines of body code. More complex lambdas should be named. A named lambda should not clutter the enclosing function and make it long and complex; do continue to break up large functions via the use of separate helper functions.</p>
+<p>An anonymous lambda expression should either be a one-liner in a one-line expression, or isolated in its own set of lines. Don't place part of a lambda expression on the same line as other arguments to a function. The body of a multi-line lambda argument should be indented from the start of the capture list, as if that were the start of an ordinary function definition. The body of a multi-line named lambda should be indented one step from the variable's indentation.</p>
+<p>Some examples:</p>
+<ol type="1">
+<li><code>foo([&amp;] { ++counter; });</code></li>
+<li><code>foo(x, [&amp;] { ++counter; });</code></li>
+<li><code>foo([&amp;] { if (predicate) ++counter; });</code></li>
+<li><code>foo([&amp;] { auto tmp = process(x); tmp.f(); return tmp.g(); })</code></li>
+<li><p>Separate one-line lambda from other arguments:</p>
+<pre><code>foo(c.begin(), c.end(),
+    [&amp;] (const X&amp; x) { do_something(x); return x.value(); });</code></pre></li>
+<li><p>Indentation for multi-line lambda:</p>
+<pre><code>c.do_entries([&amp;] (const X&amp; x) {
+               do_something(x, a);
+               do_something1(x, b);
+               do_something2(x, c);
+             });</code></pre></li>
+<li><p>Separate multi-line lambda from other arguments:</p>
+<pre><code>foo(c.begin(), c.end(),
+    [&amp;] (const X&amp; x) {
+      do_something(x, a);
+      do_something1(x, b);
+      do_something2(x, c);
+    });</code></pre></li>
+<li><p>Multi-line named lambda:</p>
+<pre><code>auto do_entry = [&amp;] (const X&amp; x) {
+  do_something(x, a);
+  do_something1(x, b);
+  do_something2(x, c);
+};</code></pre></li>
+</ol>
+<p>Item 4, and especially items 6 and 7, are pushing the simplicity limits for anonymous lambdas. Item 6 might be better written using a named lambda:</p>
+<pre><code>c.do_entries(do_entry);</code></pre>
+<p>Note that C++11 also added <em>bind expressions</em> as a way to write a function object for partial application, using <code>std::bind</code> and related facilities from the Standard Library. <code>std::bind</code> generalizes and replaces some of the binders from C++03. Bind expressions are not permitted in HotSpot code. They don't provide enough benefit over lambdas or local function classes in the cases where bind expressions are applicable to warrant the introduction of yet another mechanism in this space into HotSpot code.</p>
+<p>References:</p>
+<ul>
+<li>Local and unnamed types as template parameters (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2657.htm">n2657</a>)</li>
+<li>New wording for C++0x lambdas (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2927.pdf">n2927</a>)</li>
+<li>Generalized lambda capture (init-capture) (<a href="https://isocpp.org/files/papers/N3648.html">N3648</a>)</li>
+<li>Generic (polymorphic) lambda expressions (<a href="https://isocpp.org/files/papers/N3649.html">N3649</a>)</li>
+</ul>
+<p>References from C++17</p>
+<ul>
+<li>Wording for constexpr lambda (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0170r1.pdf">p0170r1</a>)</li>
+<li>Lambda capture of *this by Value (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0018r3.html">p0018r3</a>)</li>
+</ul>
+<p>References from C++20</p>
+<ul>
+<li>Allow lambda capture [=, this] (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0409r2.html">p0409r2</a>)</li>
+<li>Familiar template syntax for generic lambdas (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0428r2.pdf">p0428r2</a>)</li>
+<li>Simplifying implicit lambda capture (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0588r1.html">p0588r1</a>)</li>
+<li>Default constructible and assignable stateless lambdas (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0624r2.pdf">p0624r2</a>)</li>
+<li>Lambdas in unevaluated contexts (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0315r4.pdf">p0315r4</a>)</li>
+<li>Allow pack expansion in lambda init-capture (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0780r2.html">p0780r2</a>) (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2095r0.html">p2095r0</a>)</li>
+<li>Deprecate implicit capture of this via [=] (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0806r2.html">p0806r2</a>)</li>
+</ul>
+<p>References from C++23</p>
+<ul>
+<li>Make () more optional for lambdas (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1102r2.html">p1102r2</a>)</li>
+</ul>
 <h3 id="additional-permitted-features">Additional Permitted Features</h3>
 <ul>
 <li><p><code>constexpr</code> (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2235.pdf">n2235</a>) (<a href="https://isocpp.org/files/papers/N3652.html">n3652</a>)</p></li>
@@ -305,7 +420,6 @@ while ( test_foo(args...) ) { // No, excess spaces around control</code></pre></
 <li><p>Dynamic initialization and destruction with concurrency (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2660.htm">n2660</a>)</p></li>
 <li><p><code>final</code> virtual specifiers for classes and virtual functions (<a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2928.htm">n2928</a>), (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3206.htm">n3206</a>), (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3272.htm">n3272</a>)</p></li>
 <li><p><code>override</code> virtual specifiers for virtual functions (<a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2928.htm">n2928</a>), (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3206.htm">n3206</a>), (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3272.htm">n3272</a>)</p></li>
-<li><p>Local and unnamed types as template parameters (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2657.htm">n2657</a>)</p></li>
 <li><p>Range-based <code>for</code> loops (<a href="http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2930.html">n2930</a>) (<a href="https://en.cppreference.com/w/cpp/language/range-for">range-for</a>)</p></li>
 </ul>
 <h3 id="excluded-features">Excluded Features</h3>
@@ -337,7 +451,6 @@ while ( test_foo(args...) ) { // No, excess spaces around control</code></pre></
 <li><p>Member initializers and aggregates (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3653.html">n3653</a>)</p></li>
 <li><p><code>[[noreturn]]</code> attribute (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2761.pdf">n2761</a>)</p></li>
 <li><p>Rvalue references and move semantics</p></li>
-<li><p>Lambdas</p></li>
 </ul>
 </body>
 </html>

doc/hotspot-style.md

@@ -409,7 +409,7 @@ Similar discussions for some other projects:
 * [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html) —
 Currently (2020) targeting C++17.
 
-* [C++11 and C++14 use in Chromium](https://chromium-cpp.appspot.com) —
+* [C++11 and C++14 use in Chromium](https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++11.md) —
 Categorizes features as allowed, banned, or to be discussed.
 
 * [llvm Coding Standards](https://llvm.org/docs/CodingStandards.html) —
@@ -596,9 +596,7 @@ use can make code much harder to understand.
 Only use if the function body has a very small number of `return`
 statements, and generally relatively little other code.
 
-* Generic lambdas.  Lambdas are not (yet) permitted.
-
-* Lambda init captures.  Lambdas are not (yet) permitted.
+* Also see [lambda expressions](#lambdaexpressions).
 
 ### Expression SFINAE
 
@@ -703,6 +701,273 @@ Some relevant sections from cppreference.com:
 Although related, the use of `std::initializer_list` remains forbidden, as
 part of the avoidance of the C++ Standard Library in HotSpot code.
 
+### Local Function Objects
+
+* Local function objects, including lambda expressions, may be used.
+* Lambda expressions must only be used as a downward value.
+* Prefer `[&]` as the capture list of a lambda expression.
+* Return type deduction for lambda expressions is permitted, and indeed encouraged.
+* An empty parameter list for a lambda expression may be elided.
+* A lambda expression must not be `mutable`.
+* Generic lambda expressions are permitted.
+* Lambda expressions should be relatively simple.
+* Anonymous lambda expressions should not overly clutter the enclosing expression.
+* An anonymous lambda expression must not be directly invoked.
+* Bind expressions are forbidden.
+
+Single-use function objects can be defined locally within a function,
+directly at the point of use.  This is an alternative to having a function
+or function object class defined at class or namespace scope.
+
+This usage was somewhat limited by C++03, which does not permit such a class
+to be used as a template parameter.  That restriction was removed by C++11
+([n2657]). Use of this feature is permitted.
+
+Many HotSpot protocols involve "function-like" objects that involve some
+named member function rather than a call operator.  For example, a function
+that performs some action on all threads might be written as
+
+```
+void do_something() {
+  struct DoSomething : public ThreadClosure {
+    virtual void do_thread(Thread* t) {
+      ... do something with t ...
+    }
+  } closure;
+  Threads::threads_do(&closure);
+}
+```
+
+HotSpot code has historically usually placed the DoSomething class at
+namespace (or sometimes class) scope.  This separates the function's code
+from its use, often to the detriment of readability.  It requires giving the
+class a globally unique name (if at namespace scope).  It also loses the
+information that the class is intended for use in exactly one place, and
+does not have any subclasses.  (However, the latter can now be indicated by
+declaring it `final`.)  Often, for simplicity, a local class will skip
+things like access control and accessor functions, giving the enclosing
+function direct access to the implementation and eliminating some
+boilerplate that might be provided if the class is in some outer (more
+accessible) scope.  On the other hand, if there is a lot of surrounding code
+in the function body or the local class is of significant size, defining it
+locally can increase clutter and reduce readability.
+
+<a name="lambdaexpressions"></a>
+C++11 added _lambda expressions_ as a new way to write a function object.
+Simple lambda expressions can be significantly more concise than a function
+object, eliminating a lot of boiler-plate.  On the other hand, a complex
+lambda expression may not provide much, if any, readability benefit compared
+to an ordinary function object.  Also, while a lambda can encapsulate a call
+to a "function-like" object, it cannot be used in place of such.
+
+A common use for local functions is as one-use [RAII] objects.  The amount
+of boilerplate for a function object class (local or not) makes such usage
+somewhat clumsy and verbose.  But with the help of a small amount of
+supporting utility code, lambdas work particularly well for this use case.
+
+Another use for local functions is [partial application][PARTIALAPP].  Again
+here, lambdas are typically much simpler and less verbose than function
+object classes.
+
+Because of these benefits, lambda expressions are permitted in HotSpot code,
+with some restrictions and usage guidance.  An anonymous lambda is one which
+is passed directly as an argument.  A named lambda is the value of a
+variable, which is its name.
+
+Lambda expressions should only be passed downward.  In particular, a lambda
+should not be returned from a function or stored in a global variable,
+whether directly or as the value of a member of some other object.  Lambda
+capture is syntactically subtle (by design), and propagating a lambda in
+such ways can easily pass references to captured values to places where they
+are no longer valid.  In particular, members of the enclosing `this` object
+are effectively captured by reference, even if the default capture is
+by-value.  For such uses-cases a function object class should be used to
+make the desired value capturing and propagation explicit.
+
+Limiting the capture list to `[&]` (implicitly capture by reference) is a
+simplifying restriction that still provides good support for HotSpot usage,
+while reducing the cases a reader must recognize and understand.
+
+* Many common lambda uses require reference capture.  Not permitting it
+would substantially reduce the utility of lambdas.
+
+* Referential transparency.  Implicit reference capture makes variable
+references in the lambda body have the same meaning they would have in the
+enclosing code.  There isn't a semantic barrier across which the meaning of
+a variable changes.
+
+* Explicit reference capture introduces significant clutter, especially when
+lambda expressions are relatively small and simple, as they should be in
+HotSpot code.
+
+* There are a number of reasons why by-value capture might be used, but for
+the most part they don't apply to HotSpot code, given other usage restrictions.
+
+    * A primary use-case for by-value capture is to support escaping uses,
+    where values captured by-reference might become invalid.  That use-case
+    doesn't apply if only downward lambdas are used.
+
+    * By-value capture can also make a lambda-local copy for mutation, which
+    requires making the lambda `mutable`; see below.
+
+    * By-value capture might be viewed as an optimization, avoiding any
+    overhead for reference capture of cheap to copy values.  But the
+    compiler can often eliminate any such overhead.
+
+    * By-value capture by a non-`mutable` lambda makes the captured values
+    const, preventing any modification by the lambda and making the captured
+    value unaffected by modifications to the outer variable.  But this only
+    applies to captured auto variables, not member variables, and is
+    inconsistent with referential transparency.
+
+* Non-capturing lambdas (with an empty capture list - `[]`) have limited
+utility.  There are cases where no captures are required (pure functions,
+for example), but if the function is small and simple then that's obvious
+anyway.
+
+* Capture initializers (a C++14 feature - [N3649]) are not permitted.
+Capture initializers inherently increase the complexity of the capture list,
+and provide little benefit over an additional in-scope local variable.
+
+The use of `mutable` lambda expressions is forbidden because there don't
+seem to be many, if any, good use-cases for them in HotSpot.  A lambda
+expression needs to be mutable in order to modify a by-value captured value.
+But with only downward lambdas, such usage seems likely to be rare and
+complicated.  It is better to use a function object class in any such cases
+that arise, rather than requiring all HotSpot developers to understand this
+relatively obscure feature.
+
+While it is possible to directly invoke an anonymous lambda expression, that
+feature should not be used, as such a form can be confusing to readers.
+Instead, name the lambda and call it by name.
+
+Some reasons to prefer a named lambda instead of an anonymous lambda are
+
+* The body contains non-trivial control flow or declarations or other nested
+constructs.
+
+* Its role in an argument list is hard to guess without examining the
+function declaration.  Give it a name that indicates its purpose.
+
+* It has an unusual capture list.
+
+* It has a complex explicit return type or parameter types.
+
+Lambda expressions, and particularly anonymous lambda expressions, should be
+simple and compact.  One-liners are good.  Anonymous lambdas should usually
+be limited to a couple lines of body code.  More complex lambdas should be
+named.  A named lambda should not clutter the enclosing function and make it
+long and complex; do continue to break up large functions via the use of
+separate helper functions.
+
+An anonymous lambda expression should either be a one-liner in a one-line
+expression, or isolated in its own set of lines.  Don't place part of a
+lambda expression on the same line as other arguments to a function.  The
+body of a multi-line lambda argument should be indented from the start of
+the capture list, as if that were the start of an ordinary function
+definition.  The body of a multi-line named lambda should be indented one
+step from the variable's indentation.
+
+Some examples:
+
+1. `foo([&] { ++counter; });`
+2. `foo(x, [&] { ++counter; });`
+3. `foo([&] { if (predicate) ++counter; });`
+4. `foo([&] { auto tmp = process(x); tmp.f(); return tmp.g(); })`
+5. Separate one-line lambda from other arguments:
+
+    ```
+    foo(c.begin(), c.end(),
+        [&] (const X& x) { do_something(x); return x.value(); });
+    ```
+6. Indentation for multi-line lambda:
+
+    ```
+    c.do_entries([&] (const X& x) {
+                   do_something(x, a);
+                   do_something1(x, b);
+                   do_something2(x, c);
+                 });
+    ```
+7. Separate multi-line lambda from other arguments:
+
+    ```
+    foo(c.begin(), c.end(),
+        [&] (const X& x) {
+          do_something(x, a);
+          do_something1(x, b);
+          do_something2(x, c);
+        });
+    ```
+8. Multi-line named lambda:
+
+    ```
+    auto do_entry = [&] (const X& x) {
+      do_something(x, a);
+      do_something1(x, b);
+      do_something2(x, c);
+    };
+    ```
+
+Item 4, and especially items 6 and 7, are pushing the simplicity limits for
+anonymous lambdas.  Item 6 might be better written using a named lambda:
+```
+c.do_entries(do_entry);
+```
+
+Note that C++11 also added _bind expressions_ as a way to write a function
+object for partial application, using `std::bind` and related facilities
+from the Standard Library.  `std::bind` generalizes and replaces some of the
+binders from C++03.  Bind expressions are not permitted in HotSpot code.
+They don't provide enough benefit over lambdas or local function classes in
+the cases where bind expressions are applicable to warrant the introduction
+of yet another mechanism in this space into HotSpot code.
+
+References:
+
+* Local and unnamed types as template parameters ([n2657])
+* New wording for C++0x lambdas ([n2927])
+* Generalized lambda capture (init-capture) ([N3648])
+* Generic (polymorphic) lambda expressions ([N3649])
+
+[n2657]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2657.htm 
+[n2927]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2927.pdf
+[N3648]: https://isocpp.org/files/papers/N3648.html
+[N3649]: https://isocpp.org/files/papers/N3649.html
+
+References from C++17
+
+* Wording for constexpr lambda ([p0170r1])
+* Lambda capture of *this by Value ([p0018r3])
+
+[p0170r1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0170r1.pdf
+[p0018r3]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0018r3.html
+
+References from C++20
+
+* Allow lambda capture [=, this] ([p0409r2])
+* Familiar template syntax for generic lambdas ([p0428r2])
+* Simplifying implicit lambda capture ([p0588r1])
+* Default constructible and assignable stateless lambdas ([p0624r2])
+* Lambdas in unevaluated contexts ([p0315r4])
+* Allow pack expansion in lambda init-capture ([p0780r2]) ([p2095r0])
+* Deprecate implicit capture of this via [=] ([p0806r2])
+
+[p0409r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0409r2.html
+[p0428r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0428r2.pdf
+[p0588r1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0588r1.html
+[p0624r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0624r2.pdf
+[p0315r4]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0315r4.pdf
+[p0780r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0780r2.html
+[p2095r0]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2095r0.html
+[p0806r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0806r2.html
+
+References from C++23
+
+* Make () more optional for lambdas  ([p1102r2])
+
+[p1102r2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1102r2.html
+
 ### Additional Permitted Features
 
 * `constexpr`
@@ -757,9 +1022,6 @@ part of the avoidance of the C++ Standard Library in HotSpot code.
 ([n3206](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3206.htm)),
 ([n3272](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3272.htm))
 
-* Local and unnamed types as template parameters
-([n2657](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2657.htm))
-
 * Range-based `for` loops
 ([n2930](http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2009/n2930.html))
 ([range-for](https://en.cppreference.com/w/cpp/language/range-for))
@@ -837,9 +1099,6 @@ features that have not yet been discussed.
 
 * Rvalue references and move semantics
 
-* Lambdas
-
-
 [ADL]: https://en.cppreference.com/w/cpp/language/adl 
   "Argument Dependent Lookup"
 
@@ -854,3 +1113,6 @@ features that have not yet been discussed.
 
 [SFINAE]: https://en.cppreference.com/w/cpp/language/sfinae
   "Substitution Failure Is Not An Error"
+
+[PARTIALAPP]: https://en.wikipedia.org/wiki/Partial_application
+  "Partial Application"

doc/testing.html

@@ -27,6 +27,7 @@
 <li><a href="#configuration">Configuration</a></li>
 </ul></li>
 <li><a href="#test-selection">Test selection</a><ul>
+<li><a href="#common-test-groups">Common Test Groups</a></li>
 <li><a href="#jtreg">JTReg</a></li>
 <li><a href="#gtest">Gtest</a></li>
 <li><a href="#microbenchmarks">Microbenchmarks</a></li>
@@ -67,6 +68,19 @@ $ make exploded-test TEST=tier2</code></pre>
 <p>All functionality is available using the <code>test</code> make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, <code>test-only</code> can be used instead, which do not depend on the source and test image build.</p>
 <p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make test-tier1</code> is equivalent to <code>make test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>test TEST=&quot;x&quot;</code> solution needs to be used.</p>
 <p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
+<h3 id="common-test-groups">Common Test Groups</h3>
+<p>Ideally, all tests are run for every change but this may not be practical due to the limited testing resources, the scope of the change, etc.</p>
+<p>The source tree currently defines a few common test groups in the relevant <code>TEST.groups</code> files. There are test groups that cover a specific component, for example <code>hotspot_gc</code>. It is a good idea to look into <code>TEST.groups</code> files to get a sense what tests are relevant to a particular JDK component.</p>
+<p>Component-specific tests may miss some unintended consequences of a change, so other tests should also be run. Again, it might be impractical to run all tests, and therefore <em>tiered</em> test groups exist. Tiered test groups are not component-specific, but rather cover the significant parts of the entire JDK.</p>
+<p>Multiple tiers allow balancing test coverage and testing costs. Lower test tiers are supposed to contain the simpler, quicker and more stable tests. Higher tiers are supposed to contain progressively more thorough, slower, and sometimes less stable tests, or the tests that require special configuration.</p>
+<p>Contributors are expected to run the tests for the areas that are changed, and the first N tiers they can afford to run, but at least tier1.</p>
+<p>A brief description of the tiered test groups:</p>
+<ul>
+<li><p><code>tier1</code>: This is the lowest test tier. Multiple developers run these tests every day. Because of the widespread use, the tests in <code>tier1</code> are carefully selected and optimized to run fast, and to run in the most stable manner. The test failures in <code>tier1</code> are usually followed up on quickly, either with fixes, or adding relevant tests to problem list. GitHub Actions workflows, if enabled, run <code>tier1</code> tests.</p></li>
+<li><p><code>tier2</code>: This test group covers even more ground. These contain, among other things, tests that either run for too long to be at <code>tier1</code>, or may require special configuration, or tests that are less stable, or cover the broader range of non-core JVM and JDK features/components (for example, XML).</p></li>
+<li><p><code>tier3</code>: This test group includes more stressful tests, the tests for corner cases not covered by previous tiers, plus the tests that require GUIs. As such, this suite should either be run with low concurrency (<code>TEST_JOBS=1</code>), or without headful tests (<code>JTREG_KEYWORDS=\!headful</code>), or both.</p></li>
+<li><p><code>tier4</code>: This test group includes every other test not covered by previous tiers. It includes, for example, <code>vmTestbase</code> suites for Hotspot, which run for many hours even on large machines. It also runs GUI tests, so the same <code>TEST_JOBS</code> and <code>JTREG_KEYWORDS</code> caveats apply.</p></li>
+</ul>
 <h3 id="jtreg">JTReg</h3>
 <p>JTReg tests can be selected either by picking a JTReg test group, or a selection of files or directories containing JTReg tests.</p>
 <p>JTReg test groups can be specified either without a test root, e.g. <code>:tier1</code> (or <code>tier1</code>, the initial colon is optional), or with, e.g. <code>hotspot:tier1</code>, <code>test/jdk:jdk_util</code> or <code>$(TOPDIR)/test/hotspot/jtreg:hotspot_all</code>. The test root can be specified either as an absolute path, or a path relative to the JDK top directory, or the <code>test</code> directory. For simplicity, the hotspot JTReg test root, which really is <code>hotspot/jtreg</code> can be abbreviated as just <code>hotspot</code>.</p>
@@ -215,7 +229,7 @@ TEST FAILURE</code></pre>
     JTREG=&quot;JAVA_OPTIONS=-Djdk.test.docker.image.name=ubuntu
     -Djdk.test.docker.image.version=latest&quot;</code></pre>
 <h3 id="non-us-locale">Non-US locale</h3>
-<p>If your locale is non-US, some tests are likely to fail. To work around this you can set the locale to US. On Unix platforms simply setting <code>LANG=&quot;en_US&quot;</code> in the environment before running tests should work. On Windows, setting <code>JTREG=&quot;VM_OPTIONS=-Duser.language=en -Duser.country=US&quot;</code> helps for most, but not all test cases.</p>
+<p>If your locale is non-US, some tests are likely to fail. To work around this you can set the locale to US. On Unix platforms simply setting <code>LANG=&quot;en_US&quot;</code> in the environment before running tests should work. On Windows or MacOS, setting <code>JTREG=&quot;VM_OPTIONS=-Duser.language=en -Duser.country=US&quot;</code> helps for most, but not all test cases.</p>
 <p>For example:</p>
 <pre><code>$ export LANG=&quot;en_US&quot; &amp;&amp; make test TEST=...
 $ make test JTREG=&quot;VM_OPTIONS=-Duser.language=en -Duser.country=US&quot; TEST=...</code></pre>

doc/testing.md

@@ -64,6 +64,52 @@ jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1`. You can
 always submit a list of fully qualified test descriptors in the `TEST` variable
 if you want to shortcut the parser.
 
+### Common Test Groups
+
+Ideally, all tests are run for every change but this may not be practical due to the limited
+testing resources, the scope of the change, etc.
+
+The source tree currently defines a few common test groups in the relevant `TEST.groups`
+files. There are test groups that cover a specific component, for example `hotspot_gc`.
+It is a good idea to look into `TEST.groups` files to get a sense what tests are relevant
+to a particular JDK component.
+
+Component-specific tests may miss some unintended consequences of a change, so other
+tests should also be run. Again, it might be impractical to run all tests, and therefore
+_tiered_ test groups exist. Tiered test groups are not component-specific, but rather cover
+the significant parts of the entire JDK.
+
+Multiple tiers allow balancing test coverage and testing costs. Lower test tiers are supposed to
+contain the simpler, quicker and more stable tests. Higher tiers are supposed to contain
+progressively more thorough, slower, and sometimes less stable tests, or the tests that require
+special configuration.
+
+Contributors are expected to run the tests for the areas that are changed, and the first N tiers
+they can afford to run, but at least tier1.
+
+A brief description of the tiered test groups:
+
+- `tier1`: This is the lowest test tier. Multiple developers run these tests every day.
+Because of the widespread use, the tests in `tier1` are carefully selected and optimized to run
+fast, and to run in the most stable manner. The test failures in `tier1` are usually followed up
+on quickly, either with fixes, or adding relevant tests to problem list. GitHub Actions workflows,
+if enabled, run `tier1` tests.
+
+- `tier2`: This test group covers even more ground. These contain, among other things,
+tests that either run for too long to be at `tier1`, or may require special configuration,
+or tests that are less stable, or cover the broader range of non-core JVM and JDK features/components
+(for example, XML).
+
+- `tier3`: This test group includes more stressful tests, the tests for corner cases
+not covered by previous tiers, plus the tests that require GUIs. As such, this suite
+should either be run with low concurrency (`TEST_JOBS=1`), or without headful tests
+(`JTREG_KEYWORDS=\!headful`), or both.
+
+- `tier4`: This test group includes every other test not covered by previous tiers. It includes,
+for example, `vmTestbase` suites for Hotspot, which run for many hours even on large
+machines. It also runs GUI tests, so the same `TEST_JOBS` and `JTREG_KEYWORDS` caveats
+apply.
+
 ### JTReg
 
 JTReg tests can be selected either by picking a JTReg test group, or a selection
@@ -460,7 +506,7 @@ $ make test TEST="jtreg:test/hotspot/jtreg/containers/docker" \
 
 If your locale is non-US, some tests are likely to fail. To work around this
 you can set the locale to US. On Unix platforms simply setting `LANG="en_US"`
-in the environment before running tests should work. On Windows, setting
+in the environment before running tests should work. On Windows or MacOS, setting
 `JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US"` helps for most, but
 not all test cases.
 

make/Hsdis.gmk

@@ -0,0 +1,118 @@
+#
+# Copyright (c) 2020, 2021, Oracle and/or its affiliates. All rights reserved.
+# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+#
+# This code is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 2 only, as
+# published by the Free Software Foundation.  Oracle designates this
+# particular file as subject to the "Classpath" exception as provided
+# by Oracle in the LICENSE file that accompanied this code.
+#
+# This code is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# version 2 for more details (a copy is included in the LICENSE file that
+# accompanied this code).
+#
+# You should have received a copy of the GNU General Public License version
+# 2 along with this work; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+# or visit www.oracle.com if you need additional information or have any
+# questions.
+#
+
+default: all
+
+include $(SPEC)
+include MakeBase.gmk
+include JdkNativeCompilation.gmk
+
+################################################################################
+# This makefile compiles and installs the hsdis library
+#
+################################################################################
+
+HSDIS_OUTPUT_DIR := $(SUPPORT_OUTPUTDIR)/hsdis
+
+ifeq ($(call isTargetOs, windows), true)
+  INSTALLED_HSDIS_DIR := $(JDK_OUTPUTDIR)/bin
+
+  # On windows, we need to "fake" a completely different toolchain using gcc
+  # instead of the normal microsoft toolchain. This is quite hacky...
+
+  MINGW_BASE := x86_64-w64-mingw32
+
+  $(eval $(call DefineNativeToolchain, TOOLCHAIN_MINGW, \
+      CC := $(MINGW_BASE)-gcc, \
+      LD := $(MINGW_BASE)-ld, \
+      OBJCOPY := $(MINGW_BASE)-objcopy, \
+      RC := $(RC), \
+      SYSROOT_CFLAGS := --sysroot=/usr/$(MINGW_BASE)/sys-root, \
+      SYSROOT_LDFLAGS := --sysroot=/usr/$(MINGW_BASE)/sys-root, \
+  ))
+
+  TOOLCHAIN_TYPE := gcc
+  OPENJDK_TARGET_OS := linux
+  CC_OUT_OPTION := -o$(SPACE)
+  LD_OUT_OPTION := -o$(SPACE)
+  GENDEPS_FLAGS := -MMD -MF
+  CFLAGS_DEBUG_SYMBOLS := -g
+  DISABLED_WARNINGS :=
+  DISABLE_WARNING_PREFIX := -Wno-
+  CFLAGS_WARNINGS_ARE_ERRORS := -Werror
+  SHARED_LIBRARY_FLAGS := -shared
+
+  HSDIS_TOOLCHAIN := TOOLCHAIN_MINGW
+  HSDIS_TOOLCHAIN_CFLAGS :=
+  HSDIS_TOOLCHAIN_LDFLAGS := -L/usr/lib/gcc/$(MINGW_BASE)/9.2.0 \
+      -L/usr/$(MINGW_BASE)/sys-root/mingw/lib
+  MINGW_DLLCRT := /usr/$(MINGW_BASE)/sys-root/mingw/lib/dllcrt2.o
+  HSDIS_TOOLCHAIN_LIBS := $(MINGW_DLLCRT) -lmingw32 -lgcc -lgcc_eh -lmoldname \
+      -lmingwex -lmsvcrt -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32
+else
+  INSTALLED_HSDIS_DIR := $(JDK_OUTPUTDIR)/lib
+
+  HSDIS_TOOLCHAIN := TOOLCHAIN_DEFAULT
+  HSDIS_TOOLCHAIN_CFLAGS := $(CFLAGS_JDKLIB)
+  HSDIS_TOOLCHAIN_LDFLAGS := $(LDFLAGS_JDKLIB)
+  HSDIS_TOOLCHAIN_LIBS := -ldl
+endif
+
+
+$(eval $(call SetupJdkLibrary, BUILD_HSDIS, \
+    NAME := hsdis, \
+    SRC := $(TOPDIR)/src/utils/hsdis, \
+    TOOLCHAIN := $(HSDIS_TOOLCHAIN), \
+    OUTPUT_DIR := $(HSDIS_OUTPUT_DIR), \
+    OBJECT_DIR := $(HSDIS_OUTPUT_DIR), \
+    DISABLED_WARNINGS_gcc := undef format-nonliteral sign-compare, \
+    DISABLED_WARNINGS_clang := undef format-nonliteral, \
+    CFLAGS := $(HSDIS_TOOLCHAIN_CFLAGS) $(HSDIS_CFLAGS), \
+    LDFLAGS := $(HSDIS_TOOLCHAIN_LDFLAGS) $(SHARED_LIBRARY_FLAGS), \
+    LIBS := $(HSDIS_LIBS) $(HSDIS_TOOLCHAIN_LIBS), \
+))
+
+build: $(BUILD_HSDIS)
+
+TARGETS += build
+
+INSTALLED_HSDIS_NAME := hsdis-$(OPENJDK_TARGET_CPU_LEGACY_LIB)$(SHARED_LIBRARY_SUFFIX)
+
+INSTALLED_HSDIS := $(INSTALLED_HSDIS_DIR)/$(INSTALLED_HSDIS_NAME)
+
+$(INSTALLED_HSDIS): $(BUILD_HSDIS_TARGET)
+	$(call LogWarn, NOTE: The resulting build might not be redistributable. Seek legal advice before distibuting.)
+	$(install-file)
+
+
+install: $(INSTALLED_HSDIS)
+
+TARGETS += install
+
+################################################################################
+
+all: $(TARGETS)
+
+.PHONY: all default build install

make/Main.gmk

@@ -526,6 +526,18 @@ $(eval $(call SetupTarget, update-x11wrappers, \
     DEPS := java.base-copy buildtools-jdk, \
 ))
 
+ifneq ($(HSDIS_BACKEND), none)
+  $(eval $(call SetupTarget, build-hsdis, \
+      MAKEFILE := Hsdis, \
+      TARGET := build, \
+  ))
+
+  $(eval $(call SetupTarget, install-hsdis, \
+      MAKEFILE := Hsdis, \
+      TARGET := install, \
+  ))
+endif
+
 ################################################################################
 # Cross compilation support
 

make/RunTests.gmk

@@ -200,9 +200,10 @@ $(eval $(call SetTestOpt,FAILURE_HANDLER_TIMEOUT,JTREG))
 $(eval $(call ParseKeywordVariable, JTREG, \
     SINGLE_KEYWORDS := JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT \
         TEST_MODE ASSERT VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS \
-        RETRY_COUNT MAX_OUTPUT, \
+        RETRY_COUNT MAX_OUTPUT $(CUSTOM_JTREG_SINGLE_KEYWORDS), \
     STRING_KEYWORDS := OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS \
-        EXTRA_PROBLEM_LISTS LAUNCHER_OPTIONS, \
+        EXTRA_PROBLEM_LISTS LAUNCHER_OPTIONS \
+        $(CUSTOM_JTREG_STRING_KEYWORDS), \
 ))
 
 ifneq ($(JTREG), )
@@ -832,6 +833,8 @@ define SetupRunJtregTestBody
     endif
   endif
 
+  $$(eval $$(call SetupRunJtregTestCustom, $1))
+
   clean-workdir-$1:
 	$$(RM) -r $$($1_TEST_SUPPORT_DIR)
 

make/autoconf/configure.ac

@@ -249,6 +249,7 @@ JDKOPT_EXCLUDE_TRANSLATIONS
 JDKOPT_ENABLE_DISABLE_MANPAGES
 JDKOPT_ENABLE_DISABLE_CDS_ARCHIVE
 JDKOPT_ENABLE_DISABLE_COMPATIBLE_CDS_ALIGNMENT
+JDKOPT_SETUP_HSDIS
 
 ###############################################################################
 #

make/autoconf/flags-ldflags.m4

@@ -32,16 +32,10 @@ AC_DEFUN([FLAGS_SETUP_LDFLAGS],
 
   # Setup the target toolchain
 
+  # The target dir matches the name of VM variant
+  TARGET_JVM_VARIANT_PATH=$JVM_VARIANT_MAIN
+
   # On some platforms (mac) the linker warns about non existing -L dirs.
-  # For any of the variants server, client or minimal, the dir matches the
-  # variant name. The "main" variant should be used for linking. For the
-  # rest, the dir is just server.
-  if HOTSPOT_CHECK_JVM_VARIANT(server) || HOTSPOT_CHECK_JVM_VARIANT(client) \
-      || HOTSPOT_CHECK_JVM_VARIANT(minimal); then
-    TARGET_JVM_VARIANT_PATH=$JVM_VARIANT_MAIN
-  else
-    TARGET_JVM_VARIANT_PATH=server
-  fi
   FLAGS_SETUP_LDFLAGS_CPU_DEP([TARGET])
 
   # Setup the build toolchain

make/autoconf/help.m4

@@ -80,6 +80,14 @@ cygwin_help() {
       PKGHANDLER_COMMAND="( cd <location of cygwin setup.exe> && cmd /c setup -q -P make )"
       HELP_MSG="You might be able to fix this by running '$PKGHANDLER_COMMAND'."
       ;;
+    i686-w64-mingw32-gcc)
+      PKGHANDLER_COMMAND="( cd <location of cygwin setup.exe> && cmd /c setup -q -P gcc-core i686-w64-mingw32-gcc-core mingw64-i686-glib2.0 )"
+      HELP_MSG="You might be able to fix this by running '$PKGHANDLER_COMMAND'."
+      ;;
+    x86_64-w64-mingw32-gcc)
+      PKGHANDLER_COMMAND="( cd <location of cygwin setup.exe> && cmd /c setup -q -P gcc-core x86_64-w64-mingw32-gcc-core mingw64-x86_64-glib2.0 )"
+      HELP_MSG="You might be able to fix this by running '$PKGHANDLER_COMMAND'."
+      ;;
   esac
 }
 

make/autoconf/hotspot.m4

@@ -83,19 +83,10 @@ AC_DEFUN_ONCE([HOTSPOT_SETUP_JVM_VARIANTS],
     AC_MSG_ERROR([Cannot continue])
   fi
 
-  # All "special" variants share the same output directory ("server")
-  VALID_MULTIPLE_JVM_VARIANTS="server client minimal"
-  UTIL_GET_NON_MATCHING_VALUES(INVALID_MULTIPLE_VARIANTS, $JVM_VARIANTS, \
-      $VALID_MULTIPLE_JVM_VARIANTS)
-  if  test "x$INVALID_MULTIPLE_VARIANTS" != x && \
-      test "x$BUILDING_MULTIPLE_JVM_VARIANTS" = xtrue; then
-    AC_MSG_ERROR([You can only build multiple variants using these variants: '$VALID_MULTIPLE_JVM_VARIANTS'])
-  fi
-
   # The "main" variant is the one used by other libs to link against during the
   # build.
   if test "x$BUILDING_MULTIPLE_JVM_VARIANTS" = "xtrue"; then
-    MAIN_VARIANT_PRIO_ORDER="server client minimal"
+    MAIN_VARIANT_PRIO_ORDER="server client minimal zero"
     for variant in $MAIN_VARIANT_PRIO_ORDER; do
       if HOTSPOT_CHECK_JVM_VARIANT($variant); then
         JVM_VARIANT_MAIN="$variant"

make/autoconf/jdk-options.m4

@@ -704,3 +704,145 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_REPRODUCIBLE_BUILD],
   AC_SUBST(SOURCE_DATE)
   AC_SUBST(ENABLE_REPRODUCIBLE_BUILD)
 ])
+
+################################################################################
+#
+# Helper function to build binutils from source.
+#
+AC_DEFUN([JDKOPT_BUILD_BINUTILS],
+[
+  BINUTILS_SRC="$with_binutils_src"
+  UTIL_FIXUP_PATH(BINUTILS_SRC)
+
+  if ! test -d $BINUTILS_SRC; then
+    AC_MSG_ERROR([--with-binutils-src is not pointing to a directory])
+  fi
+  if ! test -x $BINUTILS_SRC/configure; then
+    AC_MSG_ERROR([--with-binutils-src does not look like a binutils source directory])
+  fi
+
+  if test -e $BINUTILS_SRC/bfd/libbfd.a && \
+      test -e $BINUTILS_SRC/opcodes/libopcodes.a && \
+      test -e $BINUTILS_SRC/libiberty/libiberty.a && \
+      test -e $BINUTILS_SRC/zlib/libz.a; then
+    AC_MSG_NOTICE([Found binutils binaries in binutils source directory -- not building])
+  else
+    # On Windows, we cannot build with the normal Microsoft CL, but must instead use
+    # a separate mingw toolchain.
+    if test "x$OPENJDK_BUILD_OS" = xwindows; then
+      if test "x$OPENJDK_TARGET_CPU" = "xx86"; then
+        target_base="i686-w64-mingw32"
+      else
+        target_base="$OPENJDK_TARGET_CPU-w64-mingw32"
+      fi
+      binutils_cc="$target_base-gcc"
+      binutils_target="--host=$target_base --target=$target_base"
+      # Somehow the uint typedef is not included when building with mingw
+      binutils_cflags="-Duint=unsigned"
+      compiler_version=`$binutils_cc --version 2>&1`
+      if ! [ [[ "$compiler_version" =~ GCC ]] ]; then
+        AC_MSG_NOTICE([Could not find correct mingw compiler $binutils_cc.])
+        HELP_MSG_MISSING_DEPENDENCY([$binutils_cc])
+        AC_MSG_ERROR([Cannot continue. $HELP_MSG])
+      else
+        AC_MSG_NOTICE([Using compiler $binutils_cc with version $compiler_version])
+      fi
+    elif test "x$OPENJDK_BUILD_OS" = xmacosx; then
+      if test "x$OPENJDK_TARGET_CPU" = "xaarch64"; then
+        binutils_target="--enable-targets=aarch64-darwin"
+      else
+        binutils_target=""
+      fi
+    else
+      binutils_cc="$CC $SYSROOT_CFLAGS"
+      binutils_target=""
+    fi
+    binutils_cflags="$binutils_cflags $MACHINE_FLAG $JVM_PICFLAG $C_O_FLAG_NORM"
+
+    AC_MSG_NOTICE([Running binutils configure])
+    AC_MSG_NOTICE([configure command line: ./configure --disable-nls CFLAGS="$binutils_cflags" CC="$binutils_cc" $binutils_target])
+    saved_dir=`pwd`
+    cd "$BINUTILS_SRC"
+    ./configure --disable-nls CFLAGS="$binutils_cflags" CC="$binutils_cc" $binutils_target
+    if test $? -ne 0 || ! test -e $BINUTILS_SRC/Makefile; then
+      AC_MSG_NOTICE([Automatic building of binutils failed on configure. Try building it manually])
+      AC_MSG_ERROR([Cannot continue])
+    fi
+    AC_MSG_NOTICE([Running binutils make])
+    $MAKE all-opcodes
+    if test $? -ne 0; then
+      AC_MSG_NOTICE([Automatic building of binutils failed on make. Try building it manually])
+      AC_MSG_ERROR([Cannot continue])
+    fi
+    cd $saved_dir
+    AC_MSG_NOTICE([Building of binutils done])
+  fi
+
+  BINUTILS_DIR="$BINUTILS_SRC"
+])
+
+################################################################################
+#
+# Determine if hsdis should be built, and if so, with which backend.
+#
+AC_DEFUN_ONCE([JDKOPT_SETUP_HSDIS],
+[
+  AC_ARG_WITH([hsdis], [AS_HELP_STRING([--with-hsdis],
+      [what hsdis backend to use ('none', 'binutils') @<:@none@:>@])])
+
+  AC_ARG_WITH([binutils], [AS_HELP_STRING([--with-binutils],
+      [where to find the binutils files needed for hsdis/binutils])])
+
+  AC_ARG_WITH([binutils-src], [AS_HELP_STRING([--with-binutils-src],
+      [where to find the binutils source for building])])
+
+  AC_MSG_CHECKING([what hsdis backend to use])
+
+  if test "x$with_hsdis" = xyes; then
+    AC_MSG_ERROR([--with-hsdis must have a value])
+  elif test "x$with_hsdis" = xnone || test "x$with_hsdis" = xno || test "x$with_hsdis" = x; then
+    HSDIS_BACKEND=none
+    AC_MSG_RESULT(['none', hsdis will not be built])
+  elif test "x$with_hsdis" = xbinutils; then
+    HSDIS_BACKEND=binutils
+    AC_MSG_RESULT(['binutils'])
+
+    # We need the binutils static libs and includes.
+    if test "x$with_binutils_src" != x; then
+      # Try building the source first. If it succeeds, it sets $BINUTILS_DIR.
+      JDKOPT_BUILD_BINUTILS
+    fi
+
+    if test "x$with_binutils" != x; then
+      BINUTILS_DIR="$with_binutils"
+    fi
+
+    AC_MSG_CHECKING([for binutils to use with hsdis])
+    if test "x$BINUTILS_DIR" != x; then
+      if test -e $BINUTILS_DIR/bfd/libbfd.a && \
+          test -e $BINUTILS_DIR/opcodes/libopcodes.a && \
+          test -e $BINUTILS_DIR/libiberty/libiberty.a; then
+        AC_MSG_RESULT([$BINUTILS_DIR])
+        HSDIS_CFLAGS="-I$BINUTILS_DIR/include -I$BINUTILS_DIR/bfd -DLIBARCH_$OPENJDK_TARGET_CPU_LEGACY_LIB"
+        HSDIS_LIBS="$BINUTILS_DIR/bfd/libbfd.a $BINUTILS_DIR/opcodes/libopcodes.a $BINUTILS_DIR/libiberty/libiberty.a $BINUTILS_DIR/zlib/libz.a"
+      else
+        AC_MSG_RESULT([invalid])
+        AC_MSG_ERROR([$BINUTILS_DIR does not contain a proper binutils installation])
+      fi
+    else
+      AC_MSG_RESULT([missing])
+      AC_MSG_NOTICE([--with-hsdis=binutils requires specifying a binutils installation.])
+      AC_MSG_NOTICE([Download binutils from https://www.gnu.org/software/binutils and unpack it,])
+      AC_MSG_NOTICE([and point --with-binutils-src to the resulting directory, or use])
+      AC_MSG_NOTICE([--with-binutils to point to a pre-built binutils installation.])
+      AC_MSG_ERROR([Cannot continue])
+    fi
+  else
+    AC_MSG_RESULT([invalid])
+    AC_MSG_ERROR([Incorrect hsdis backend "$with_hsdis"])
+  fi
+
+  AC_SUBST(HSDIS_BACKEND)
+  AC_SUBST(HSDIS_CFLAGS)
+  AC_SUBST(HSDIS_LIBS)
+])

make/autoconf/jvm-features.m4

@@ -357,6 +357,13 @@ AC_DEFUN_ONCE([JVM_FEATURES_CHECK_ZGC],
         AC_MSG_RESULT([no, $OPENJDK_TARGET_OS-$OPENJDK_TARGET_CPU])
         AVAILABLE=false
       fi
+    elif test "x$OPENJDK_TARGET_CPU" = "xppc64le"; then
+      if test "x$OPENJDK_TARGET_OS" = "xlinux"; then
+        AC_MSG_RESULT([yes])
+      else
+        AC_MSG_RESULT([no, $OPENJDK_TARGET_OS-$OPENJDK_TARGET_CPU])
+        AVAILABLE=false
+      fi
     else
       AC_MSG_RESULT([no, $OPENJDK_TARGET_OS-$OPENJDK_TARGET_CPU])
       AVAILABLE=false

make/autoconf/spec.gmk.in

@@ -359,6 +359,10 @@ ENABLE_COMPATIBLE_CDS_ALIGNMENT := @ENABLE_COMPATIBLE_CDS_ALIGNMENT@
 
 ALLOW_ABSOLUTE_PATHS_IN_OUTPUT := @ALLOW_ABSOLUTE_PATHS_IN_OUTPUT@
 
+HSDIS_BACKEND := @HSDIS_BACKEND@
+HSDIS_CFLAGS := @HSDIS_CFLAGS@
+HSDIS_LIBS := @HSDIS_LIBS@
+
 # The boot jdk to use. This is overridden in bootcycle-spec.gmk. Make sure to keep
 # it in sync.
 BOOT_JDK:=@BOOT_JDK@

make/autoconf/toolchain.m4

@@ -51,7 +51,7 @@ TOOLCHAIN_DESCRIPTION_xlc="IBM XL C/C++"
 
 # Minimum supported versions, empty means unspecified
 TOOLCHAIN_MINIMUM_VERSION_clang="3.5"
-TOOLCHAIN_MINIMUM_VERSION_gcc="5.0"
+TOOLCHAIN_MINIMUM_VERSION_gcc="6.0"
 TOOLCHAIN_MINIMUM_VERSION_microsoft="19.10.0.0" # VS2017
 TOOLCHAIN_MINIMUM_VERSION_xlc=""
 

make/conf/jib-profiles.js

@@ -249,7 +249,7 @@ var getJibProfilesCommon = function (input, data) {
         dependencies: ["boot_jdk", "gnumake", "jtreg", "jib", "autoconf", "jmh", "jcov"],
         default_make_targets: ["product-bundles", "test-bundles", "static-libs-bundles"],
         configure_args: concat("--enable-jtreg-failure-handler",
-            "--with-exclude-translations=de,es,fr,it,ko,pt_BR,sv,ca,tr,cs,sk,ja_JP_A,ja_JP_HA,ja_JP_HI,ja_JP_I,zh_TW,zh_HK",
+            "--with-exclude-translations=es,fr,it,ko,pt_BR,sv,ca,tr,cs,sk,ja_JP_A,ja_JP_HA,ja_JP_HI,ja_JP_I,zh_TW,zh_HK",
             "--disable-manpages",
             "--disable-jvm-feature-shenandoahgc",
             versionArgs(input, common))
@@ -394,13 +394,8 @@ var getJibProfilesCommon = function (input, data) {
         };
     };
 
-    if (input.build_os == 'macosx' && input.build_cpu == 'aarch64') {
-        common.boot_jdk_version = "17";
-        common.boot_jdk_build_number = "24";
-    } else {
-        common.boot_jdk_version = "16";
-        common.boot_jdk_build_number = "36";
-    }
+    common.boot_jdk_version = "17";
+    common.boot_jdk_build_number = "35";
     common.boot_jdk_home = input.get("boot_jdk", "install_path") + "/jdk-"
         + common.boot_jdk_version
         + (input.build_os == "macosx" ? ".jdk/Contents/Home" : "");
@@ -638,7 +633,7 @@ var getJibProfilesProfiles = function (input, common, data) {
     // Bootcycle profiles runs the build with itself as the boot jdk. This can
     // be done in two ways. Either using the builtin bootcycle target in the
     // build system. Or by supplying the main jdk build as bootjdk to configure.
-    [ "linux-x64", "macosx-x64", "windows-x64" ]
+    [ "linux-x64", "macosx-x64", "windows-x64", "linux-aarch64" ]
         .forEach(function (name) {
             var bootcycleName = name + "-bootcycle";
             var bootcyclePrebuiltName = name + "-bootcycle-prebuilt";
@@ -1073,11 +1068,7 @@ var getJibProfilesDependencies = function (input, common) {
     }
     var boot_jdk_os = input.build_os;
     if (input.build_os == "macosx") {
-        if (input.build_cpu == "aarch64") {
-            boot_jdk_os = "macos";
-        } else {
-            boot_jdk_os = "osx";
-        }
+        boot_jdk_os = "macos";
     }
     var boot_jdk_platform = boot_jdk_os + "-" + input.build_cpu;
     var boot_jdk_ext = (input.build_os == "windows" ? ".zip" : ".tar.gz")

make/conf/test-dependencies

@@ -25,19 +25,19 @@
 
 # Versions and download locations for dependencies used by pre-submit testing.
 
-BOOT_JDK_VERSION=16
+BOOT_JDK_VERSION=17
 JTREG_VERSION=6
 JTREG_BUILD=1
 GTEST_VERSION=1.8.1
 
-LINUX_X64_BOOT_JDK_FILENAME=openjdk-16_linux-x64_bin.tar.gz
-LINUX_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk16/7863447f0ab643c585b9bdebf67c69db/36/GPL/openjdk-16_linux-x64_bin.tar.gz
-LINUX_X64_BOOT_JDK_SHA256=e952958f16797ad7dc7cd8b724edd69ec7e0e0434537d80d6b5165193e33b931
+LINUX_X64_BOOT_JDK_FILENAME=openjdk-17_linux-x64_bin.tar.gz
+LINUX_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk17/0d483333a00540d886896bac774ff48b/35/GPL/openjdk-17_linux-x64_bin.tar.gz
+LINUX_X64_BOOT_JDK_SHA256=aef49cc7aa606de2044302e757fa94c8e144818e93487081c4fd319ca858134b
 
-WINDOWS_X64_BOOT_JDK_FILENAME=openjdk-16_windows-x64_bin.zip
-WINDOWS_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk16/7863447f0ab643c585b9bdebf67c69db/36/GPL/openjdk-16_windows-x64_bin.zip
-WINDOWS_X64_BOOT_JDK_SHA256=a78bdeaad186297601edac6772d931224d7af6f682a43372e693c37020bd37d6
+WINDOWS_X64_BOOT_JDK_FILENAME=openjdk-17_windows-x64_bin.zip
+WINDOWS_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk17/0d483333a00540d886896bac774ff48b/35/GPL/openjdk-17_windows-x64_bin.zip
+WINDOWS_X64_BOOT_JDK_SHA256=e88b0df00021c9d266bb435c9a95fdc67d1948cce4518daf85c234907bd393c5
 
-MACOS_X64_BOOT_JDK_FILENAME=openjdk-16_osx-x64_bin.tar.gz
-MACOS_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk16/7863447f0ab643c585b9bdebf67c69db/36/GPL/openjdk-16_osx-x64_bin.tar.gz
-MACOS_X64_BOOT_JDK_SHA256=16f3e39a31e86f3f51b0b4035a37494a47ed3c4ead760eafc6afd7afdf2ad9f2
+MACOS_X64_BOOT_JDK_FILENAME=openjdk-17_macos-x64_bin.tar.gz
+MACOS_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk17/0d483333a00540d886896bac774ff48b/35/GPL/openjdk-17_macos-x64_bin.tar.gz
+MACOS_X64_BOOT_JDK_SHA256=18e11cf9bbc6f584031e801b11ae05a233c32086f8e1b84eb8a1e9bb8e1f5d90

make/conf/version-numbers.conf

@@ -37,6 +37,6 @@ DEFAULT_VERSION_DATE=2022-03-15
 DEFAULT_VERSION_CLASSFILE_MAJOR=62  # "`$EXPR $DEFAULT_VERSION_FEATURE + 44`"
 DEFAULT_VERSION_CLASSFILE_MINOR=0
 DEFAULT_VERSION_DOCS_API_SINCE=11
-DEFAULT_ACCEPTABLE_BOOT_VERSIONS="16 17 18"
+DEFAULT_ACCEPTABLE_BOOT_VERSIONS="17 18"
 DEFAULT_JDK_SOURCE_TARGET_VERSION=18
 DEFAULT_PROMOTED_VERSION_PRE=ea

make/data/currency/CurrencyData.properties

@@ -32,7 +32,7 @@ formatVersion=3
 # Version of the currency code information in this class.
 # It is a serial number that accompanies with each amendment.
 
-dataVersion=169
+dataVersion=170
 
 # List of all valid ISO 4217 currency codes.
 # To ensure compatibility, do not remove codes.
@@ -54,7 +54,7 @@ all=ADP020-AED784-AFA004-AFN971-ALL008-AMD051-ANG532-AOA973-ARS032-ATS040-AUD036
     SBD090-SCR690-SDD736-SDG938-SEK752-SGD702-SHP654-SIT705-SKK703-SLL694-SOS706-\
     SRD968-SRG740-SSP728-STD678-STN930-SVC222-SYP760-SZL748-THB764-TJS972-TMM795-TMT934-TND788-TOP776-\
     TPE626-TRL792-TRY949-TTD780-TWD901-TZS834-UAH980-UGX800-USD840-USN997-USS998-UYI940-\
-    UYU858-UZS860-VEB862-VEF937-VES928-VND704-VUV548-WST882-XAF950-XAG961-XAU959-XBA955-\
+    UYU858-UZS860-VEB862-VED926-VEF937-VES928-VND704-VUV548-WST882-XAF950-XAG961-XAU959-XBA955-\
     XBB956-XBC957-XBD958-XCD951-XDR960-XFO000-XFU000-XOF952-XPD964-XPF953-\
     XPT962-XSU994-XTS963-XUA965-XXX999-YER886-YUM891-ZAR710-ZMK894-ZMW967-ZWD716-ZWL932-\
     ZWN942-ZWR935

make/data/jdwp/jdwp.spec

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 1998, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 1998, 2021, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -134,9 +134,9 @@ JDWP "Java(tm) Debug Wire Protocol"
         "<ul>"
         "<li>All event requests are cancelled. "
         "<li>All threads suspended by the thread-level "
-        "<a href=\"#JDWP_ThreadReference_Resume\">resume</a> command "
+        "<a href=\"#JDWP_ThreadReference_Suspend\">suspend</a> command "
         "or the VM-level "
-        "<a href=\"#JDWP_VirtualMachine_Resume\">resume</a> command "
+        "<a href=\"#JDWP_VirtualMachine_Suspend\">suspend</a> command "
         "are resumed as many times as necessary for them to run. "
         "<li>Garbage collection is re-enabled in all cases where it was "
         "<a href=\"#JDWP_ObjectReference_DisableCollection\">disabled</a> "

make/data/tzdata/VERSION

@@ -21,4 +21,4 @@
 # or visit www.oracle.com if you need additional information or have any
 # questions.
 #
-tzdata2021a
+tzdata2021c

make/data/tzdata/africa

@@ -53,9 +53,6 @@
 # Milne J. Civil time. Geogr J. 1899 Feb;13(2):173-94.
 # https://www.jstor.org/stable/1774359
 #
-# A reliable and entertaining source about time zones is
-# Derek Howse, Greenwich time and longitude, Philip Wilson Publishers (1997).
-#
 # European-style abbreviations are commonly used along the Mediterranean.
 # For sub-Saharan Africa abbreviations were less standardized.
 # Previous editions of this database used WAT, CAT, SAT, and EAT
@@ -176,8 +173,9 @@ Zone	Africa/Ndjamena	1:00:12 -	LMT	1912        # N'Djamena
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Africa/Abidjan	-0:16:08 -	LMT	1912
 			 0:00	-	GMT
+Link Africa/Abidjan Africa/Accra	# Ghana
 Link Africa/Abidjan Africa/Bamako	# Mali
-Link Africa/Abidjan Africa/Banjul	# Gambia
+Link Africa/Abidjan Africa/Banjul	# The Gambia
 Link Africa/Abidjan Africa/Conakry	# Guinea
 Link Africa/Abidjan Africa/Dakar	# Senegal
 Link Africa/Abidjan Africa/Freetown	# Sierra Leone
@@ -404,93 +402,8 @@ Zone	Africa/Cairo	2:05:09 -	LMT	1900 Oct
 # Gabon
 # See Africa/Lagos.
 
-# Gambia
-# See Africa/Abidjan.
-
+# The Gambia
 # Ghana
-
-# From P Chan (2020-11-20):
-# Interpretation Amendment Ordinance, 1915 (No.24 of 1915) [1915-11-02]
-# Ordinances of the Gold Coast, Ashanti, Northern Territories 1915, p 69-71
-# https://books.google.com/books?id=ErA-AQAAIAAJ&pg=PA70
-# This Ordinance added "'Time' shall mean Greenwich Mean Time" to the
-# Interpretation Ordinance, 1876.
-#
-# Determination of the Time Ordinance, 1919 (No. 18 of 1919) [1919-11-24]
-# Ordinances of the Gold Coast, Ashanti, Northern Territories 1919, p 75-76
-# https://books.google.com/books?id=MbA-AQAAIAAJ&pg=PA75
-# This Ordinance removed the previous definition of time and introduced DST.
-#
-# Time Determination Ordinance (Cap. 214)
-# The Laws of the Gold Coast (including Togoland Under British Mandate)
-# Vol. II (1937), p 2328
-# https://books.google.com/books?id=Z7M-AQAAIAAJ&pg=PA2328
-# Revised edition of the 1919 Ordinance.
-#
-# Time Determination (Amendment) Ordinance, 1940 (No. 9 of 1940) [1940-04-06]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1940, p 22
-# https://books.google.com/books?id=1ao-AQAAIAAJ&pg=PA22
-# This Ordinance changed the forward transition from September to May.
-#
-# Defence (Time Determination Ordinance Amendment) Regulations, 1942
-# (Regulations No. 6 of 1942) [1942-01-31, commenced on 1942-02-08]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1942, p 48
-# https://books.google.com/books?id=Das-AQAAIAAJ&pg=PA48
-# These regulations advanced the [standard] time by thirty minutes.
-#
-# Defence (Time Determination Ordinance Amendment (No.2)) Regulations,
-# 1942 (Regulations No. 28 of 1942) [1942-04-25]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1942, p 87
-# https://books.google.com/books?id=Das-AQAAIAAJ&pg=PA87
-# These regulations abolished DST and changed the time to GMT+0:30.
-#
-# Defence (Revocation) (No.4) Regulations, 1945 (Regulations No. 45 of
-# 1945) [1945-10-24, commenced on 1946-01-06]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1945, p 256
-# https://books.google.com/books?id=9as-AQAAIAAJ&pg=PA256
-# These regulations revoked the previous two sets of Regulations.
-#
-# Time Determination (Amendment) Ordinance, 1945 (No. 18 of 1945) [1946-01-06]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1945, p 69
-# https://books.google.com/books?id=9as-AQAAIAAJ&pg=PA69
-# This Ordinance abolished DST.
-#
-# Time Determination (Amendment) Ordinance, 1950 (No. 26 of 1950) [1950-07-22]
-# Annual Volume of the Laws of the Gold Coast:
-# Containing All Legislation Enacted During Year 1950, p 35
-# https://books.google.com/books?id=e60-AQAAIAAJ&pg=PA35
-# This Ordinance restored DST but with thirty minutes offset.
-#
-# Time Determination Ordinance (Cap. 264)
-# The Laws of the Gold Coast, Vol. V (1954), p 380
-# https://books.google.com/books?id=Mqc-AQAAIAAJ&pg=PA380
-# Revised edition of the Time Determination Ordinance.
-#
-# Time Determination (Amendment) Ordinance, 1956 (No. 21 of 1956) [1956-08-29]
-# Annual Volume of the Ordinances of the Gold Coast Enacted During the
-# Year 1956, p 83
-# https://books.google.com/books?id=VLE-AQAAIAAJ&pg=PA83
-# This Ordinance abolished DST.
-
-# Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
-Rule	Ghana	1919	only	-	Nov	24	0:00	0:20	+0020
-Rule	Ghana	1920	1942	-	Jan	 1	2:00	0	GMT
-Rule	Ghana	1920	1939	-	Sep	 1	2:00	0:20	+0020
-Rule	Ghana	1940	1941	-	May	 1	2:00	0:20	+0020
-Rule	Ghana	1950	1955	-	Sep	 1	2:00	0:30	+0030
-Rule	Ghana	1951	1956	-	Jan	 1	2:00	0	GMT
-
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone	Africa/Accra	-0:00:52 -	LMT	1915 Nov  2
-			 0:00	Ghana	%s	1942 Feb  8
-			 0:30	-	+0030	1946 Jan  6
-			 0:00	Ghana	%s
-
 # Guinea
 # See Africa/Abidjan.
 
@@ -755,7 +668,7 @@ Zone Indian/Mauritius	3:50:00 -	LMT	1907 # Port Louis
 # See Africa/Nairobi.
 
 # Morocco
-# See the 'europe' file for Spanish Morocco (Africa/Ceuta).
+# See Africa/Ceuta for Spanish Morocco.
 
 # From Alex Krivenyshev (2008-05-09):
 # Here is an article that Morocco plan to introduce Daylight Saving Time between
@@ -1405,23 +1318,21 @@ Zone	Africa/Lagos	0:13:35 -	LMT	1905 Jul  1
 			0:13:35	-	LMT	1914 Jan  1
 			0:30	-	+0030	1919 Sep  1
 			1:00	-	WAT
-Link Africa/Lagos Africa/Bangui	     # Central African Republic
-Link Africa/Lagos Africa/Brazzaville # Rep. of the Congo
-Link Africa/Lagos Africa/Douala	     # Cameroon
-Link Africa/Lagos Africa/Kinshasa    # Dem. Rep. of the Congo (west)
-Link Africa/Lagos Africa/Libreville  # Gabon
-Link Africa/Lagos Africa/Luanda	     # Angola
-Link Africa/Lagos Africa/Malabo	     # Equatorial Guinea
-Link Africa/Lagos Africa/Niamey	     # Niger
-Link Africa/Lagos Africa/Porto-Novo  # Benin
+Link Africa/Lagos Africa/Bangui		# Central African Republic
+Link Africa/Lagos Africa/Brazzaville	# Rep. of the Congo
+Link Africa/Lagos Africa/Douala		# Cameroon
+Link Africa/Lagos Africa/Kinshasa	# Dem. Rep. of the Congo (west)
+Link Africa/Lagos Africa/Libreville	# Gabon
+Link Africa/Lagos Africa/Luanda		# Angola
+Link Africa/Lagos Africa/Malabo		# Equatorial Guinea
+Link Africa/Lagos Africa/Niamey		# Niger
+Link Africa/Lagos Africa/Porto-Novo	# Benin
 
 # Réunion
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Indian/Reunion	3:41:52 -	LMT	1911 Jun # Saint-Denis
 			4:00	-	+04
 #
-# Crozet Islands also observes Réunion time; see the 'antarctica' file.
-#
 # Scattered Islands (Îles Éparses) administered from Réunion are as follows.
 # The following information about them is taken from
 # Îles Éparses (<http://www.outre-mer.gouv.fr/domtom/ile.htm>, 1997-07-22,
@@ -1513,8 +1424,8 @@ Rule	SA	1943	1944	-	Mar	Sun>=15	2:00	0	-
 Zone Africa/Johannesburg 1:52:00 -	LMT	1892 Feb 8
 			1:30	-	SAST	1903 Mar
 			2:00	SA	SAST
-Link Africa/Johannesburg Africa/Maseru	   # Lesotho
-Link Africa/Johannesburg Africa/Mbabane    # Eswatini
+Link Africa/Johannesburg Africa/Maseru	# Lesotho
+Link Africa/Johannesburg Africa/Mbabane	# Eswatini
 #
 # Marion and Prince Edward Is
 # scientific station since 1947
@@ -1550,12 +1461,13 @@ Zone	Africa/Khartoum	2:10:08 -	LMT	1931
 			3:00	-	EAT	2017 Nov  1
 			2:00	-	CAT
 
+# South Sudan
+
 # From Steffen Thorsen (2021-01-18):
 # "South Sudan will change its time zone by setting the clock back 1
 # hour on February 1, 2021...."
 # from https://eyeradio.org/south-sudan-adopts-new-time-zone-makuei/
 
-# South Sudan
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Africa/Juba	2:06:28 -	LMT	1931
 			2:00	Sudan	CA%sT	2000 Jan 15 12:00
@@ -1660,7 +1572,7 @@ Rule	Tunisia	2005	only	-	Sep	30	 1:00s	0	-
 Rule	Tunisia	2006	2008	-	Mar	lastSun	 2:00s	1:00	S
 Rule	Tunisia	2006	2008	-	Oct	lastSun	 2:00s	0	-
 
-# See Europe/Paris for PMT-related transitions.
+# See Europe/Paris commentary for PMT-related transitions.
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Africa/Tunis	0:40:44 -	LMT	1881 May 12
 			0:09:21	-	PMT	1911 Mar 11 # Paris Mean Time

make/data/tzdata/antarctica

@@ -171,7 +171,7 @@ Zone Antarctica/Mawson	0	-	-00	1954 Feb 13
 #
 # Alfred Faure, Possession Island, Crozet Islands, -462551+0515152, since 1964;
 #	sealing & whaling stations operated variously 1802/1911+;
-#	see Indian/Reunion.
+#	see Asia/Dubai.
 #
 # Martin-de-Viviès, Amsterdam Island, -374105+0773155, since 1950
 # Port-aux-Français, Kerguelen Islands, -492110+0701303, since 1951;
@@ -185,17 +185,7 @@ Zone Indian/Kerguelen	0	-	-00	1950 # Port-aux-Français
 			5:00	-	+05
 #
 # year-round base in the main continent
-# Dumont d'Urville, Île des Pétrels, -6640+14001, since 1956-11
-# <https://en.wikipedia.org/wiki/Dumont_d'Urville_Station> (2005-12-05)
-#
-# Another base at Port-Martin, 50km east, began operation in 1947.
-# It was destroyed by fire on 1952-01-14.
-#
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone Antarctica/DumontDUrville 0 -	-00	1947
-			10:00	-	+10	1952 Jan 14
-			0	-	-00	1956 Nov
-			10:00	-	+10
+# Dumont d'Urville - see Pacific/Port_Moresby.
 
 # France & Italy - year-round base
 # Concordia, -750600+1232000, since 2005
@@ -211,20 +201,7 @@ Zone Antarctica/DumontDUrville 0 -	-00	1947
 # Zuchelli, Terra Nova Bay, -744140+1640647, since 1986
 
 # Japan - year-round bases
-# Syowa (also known as Showa), -690022+0393524, since 1957
-#
-# From Hideyuki Suzuki (1999-02-06):
-# In all Japanese stations, +0300 is used as the standard time.
-#
-# Syowa station, which is the first antarctic station of Japan,
-# was established on 1957-01-29.  Since Syowa station is still the main
-# station of Japan, it's appropriate for the principal location.
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone Antarctica/Syowa	0	-	-00	1957 Jan 29
-			3:00	-	+03
-# See:
-# NIPR Antarctic Research Activities (1999-08-17)
-# http://www.nipr.ac.jp/english/ara01.html
+# See Asia/Riyadh.
 
 # S Korea - year-round base
 # Jang Bogo, Terra Nova Bay, -743700+1641205 since 2014

make/data/tzdata/asia

@@ -57,9 +57,6 @@
 # Byalokoz EL. New Counting of Time in Russia since July 1, 1919.
 # (See the 'europe' file for a fuller citation.)
 #
-# A reliable and entertaining source about time zones is
-# Derek Howse, Greenwich time and longitude, Philip Wilson Publishers (1997).
-#
 # The following alphabetic abbreviations appear in these tables
 # (corrections are welcome):
 #	     std  dst
@@ -2257,6 +2254,14 @@ Zone	Asia/Tokyo	9:18:59	-	LMT	1887 Dec 31 15:00u
 # From Paul Eggert (2013-12-11):
 # As Steffen suggested, consider the past 21-month experiment to be DST.
 
+# From Steffen Thorsen (2021-09-24):
+# The Jordanian Government announced yesterday that they will start DST
+# in February instead of March:
+# https://petra.gov.jo/Include/InnerPage.jsp?ID=37683&lang=en&name=en_news (English)
+# https://petra.gov.jo/Include/InnerPage.jsp?ID=189969&lang=ar&name=news (Arabic)
+# From the Arabic version, it seems to say it would be at midnight
+# (assume 24:00) on the last Thursday in February, starting from 2022.
+
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
 Rule	Jordan	1973	only	-	Jun	6	0:00	1:00	S
 Rule	Jordan	1973	1975	-	Oct	1	0:00	0	-
@@ -2287,8 +2292,9 @@ Rule	Jordan	2004	only	-	Oct	15	0:00s	0	-
 Rule	Jordan	2005	only	-	Sep	lastFri	0:00s	0	-
 Rule	Jordan	2006	2011	-	Oct	lastFri	0:00s	0	-
 Rule	Jordan	2013	only	-	Dec	20	0:00	0	-
-Rule	Jordan	2014	max	-	Mar	lastThu	24:00	1:00	S
+Rule	Jordan	2014	2021	-	Mar	lastThu	24:00	1:00	S
 Rule	Jordan	2014	max	-	Oct	lastFri	0:00s	0	-
+Rule	Jordan	2022	max	-	Feb	lastThu	24:00	1:00	S
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Asia/Amman	2:23:44 -	LMT	1931
 			2:00	Jordan	EE%sT
@@ -2763,7 +2769,8 @@ Rule	NBorneo	1935	1941	-	Dec	14	0:00	0	-
 #
 # peninsular Malaysia
 # taken from Mok Ly Yng (2003-10-30)
-# http://www.math.nus.edu.sg/aslaksen/teaching/timezone.html
+# https://web.archive.org/web/20190822231045/http://www.math.nus.edu.sg/~mathelmr/teaching/timezone.html
+# This agrees with Singapore since 1905-06-01.
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone Asia/Kuala_Lumpur	6:46:46 -	LMT	1901 Jan  1
 			6:55:25	-	SMT	1905 Jun  1 # Singapore M.T.
@@ -3523,6 +3530,12 @@ Zone	Asia/Hebron	2:20:23	-	LMT	1900 Oct
 # influence of the sources.  There is no current abbreviation for DST,
 # so use "PDT", the usual American style.
 
+# From P Chan (2021-05-10):
+# Here's a fairly comprehensive article in Japanese:
+# https://wiki.suikawiki.org/n/Philippine%20Time
+# From Paul Eggert (2021-05-10):
+# The info in the Japanese table has not been absorbed (yet) below.
+
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
 Rule	Phil	1936	only	-	Nov	1	0:00	1:00	D
 Rule	Phil	1937	only	-	Feb	1	0:00	0	S
@@ -3589,12 +3602,13 @@ Link Asia/Qatar Asia/Bahrain
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Asia/Riyadh	3:06:52 -	LMT	1947 Mar 14
 			3:00	-	+03
+Link Asia/Riyadh Antarctica/Syowa
 Link Asia/Riyadh Asia/Aden	# Yemen
 Link Asia/Riyadh Asia/Kuwait
 
 # Singapore
 # taken from Mok Ly Yng (2003-10-30)
-# http://www.math.nus.edu.sg/aslaksen/teaching/timezone.html
+# https://web.archive.org/web/20190822231045/http://www.math.nus.edu.sg/~mathelmr/teaching/timezone.html
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone	Asia/Singapore	6:55:25 -	LMT	1901 Jan  1
 			6:55:25	-	SMT	1905 Jun  1 # Singapore M.T.

make/data/tzdata/australasia

@@ -487,7 +487,7 @@ Link Pacific/Guam Pacific/Saipan # N Mariana Is
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone Pacific/Tarawa	 11:32:04 -	LMT	1901 # Bairiki
 			 12:00	-	+12
-Zone Pacific/Enderbury	-11:24:20 -	LMT	1901
+Zone Pacific/Kanton	  0	-	-00	1937 Aug 31
 			-12:00	-	-12	1979 Oct
 			-11:00	-	-11	1994 Dec 31
 			 13:00	-	+13
@@ -620,13 +620,46 @@ Link Pacific/Auckland Antarctica/McMurdo
 # was probably like Pacific/Auckland
 
 # Cook Is
-# From Shanks & Pottenger:
+#
+# From Alexander Krivenyshev (2021-03-24):
+# In 1899 the Cook Islands celebrated Christmas twice to correct the calendar.
+# According to the old books, missionaries were unaware of
+# the International Date line, when they came from Sydney.
+# Thus the Cook Islands were one day ahead....
+# http://nzetc.victoria.ac.nz/tm/scholarly/tei-KloDisc-t1-body-d18.html
+# ... Appendix to the Journals of the House of Representatives, 1900
+# https://atojs.natlib.govt.nz/cgi-bin/atojs?a=d&d=AJHR1900-I.2.1.2.3
+# (page 20)
+#
+# From Michael Deckers (2021-03-24):
+# ... in the Cook Island Act of 1915-10-11, online at
+# http://www.paclii.org/ck/legis/ck-nz_act/cia1915132/
+# "651. The hour of the day shall in each of the islands included in the
+#  Cook Islands be determined in accordance with the meridian of that island."
+# so that local (mean?) time was still used in Rarotonga (and Niue) in 1915.
+# This was changed in the Cook Island Amendment Act of 1952-10-16 ...
+# http://www.paclii.org/ck/legis/ck-nz_act/ciaa1952212/
+# "651 (1) The hour of the day in each of the islands included in the Cook
+#  Islands, other than Niue, shall be determined as if each island were
+#  situated on the meridian one hundred and fifty-seven degrees thirty minutes
+#  West of Greenwich.  (2) The hour of the day in the Island of Niue shall be
+#  determined as if that island were situated on the meridian one hundred and
+#  seventy degrees West of Greenwich."
+# This act does not state when it takes effect, so one has to assume it
+# applies since 1952-10-16.  But there is the possibility that the act just
+# legalized prior existing practice, as we had seen with the Guernsey law of
+# 1913-06-18 for the switch in 1909-04-19.
+#
+# From Paul Eggert (2021-03-24):
+# Transitions after 1952 are from Shanks & Pottenger.
+#
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
 Rule	Cook	1978	only	-	Nov	12	0:00	0:30	-
 Rule	Cook	1979	1991	-	Mar	Sun>=1	0:00	0	-
 Rule	Cook	1979	1990	-	Oct	lastSun	0:00	0:30	-
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone Pacific/Rarotonga	-10:39:04 -	LMT	1901        # Avarua
+Zone Pacific/Rarotonga	13:20:56 -	LMT	1899 Dec 26 # Avarua
+			-10:39:04 -	LMT	1952 Oct 16
 			-10:30	-	-1030	1978 Nov 12
 			-10:00	Cook	-10/-0930
 
@@ -634,10 +667,18 @@ Zone Pacific/Rarotonga	-10:39:04 -	LMT	1901        # Avarua
 
 
 # Niue
+# See Pacific/Raratonga comments for 1952 transition.
+#
+# From Tim Parenti (2021-09-13):
+# Consecutive contemporaneous editions of The Air Almanac listed -11:20 for
+# Niue as of Apr 1964 but -11 as of Aug 1964:
+#   Apr 1964: https://books.google.com/books?id=_1So677Y5vUC&pg=SL1-PA23
+#   Aug 1964: https://books.google.com/books?id=MbJloqd-zyUC&pg=SL1-PA23
+# Without greater specificity, guess 1964-07-01 for this transition.
+
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone	Pacific/Niue	-11:19:40 -	LMT	1901        # Alofi
-			-11:20	-	-1120	1951
-			-11:30	-	-1130	1978 Oct  1
+Zone	Pacific/Niue	-11:19:40 -	LMT	1952 Oct 16	# Alofi
+			-11:20	-	-1120	1964 Jul
 			-11:00	-	-11
 
 # Norfolk
@@ -661,6 +702,7 @@ Zone Pacific/Palau	-15:02:04 -	LMT	1844 Dec 31	# Koror
 Zone Pacific/Port_Moresby 9:48:40 -	LMT	1880
 			9:48:32	-	PMMT	1895 # Port Moresby Mean Time
 			10:00	-	+10
+Link Pacific/Port_Moresby Antarctica/DumontDUrville
 #
 # From Paul Eggert (2014-10-13):
 # Base the Bougainville entry on the Arawa-Kieta region, which appears to have
@@ -765,13 +807,17 @@ Link Pacific/Pago_Pago Pacific/Midway # in US minor outlying islands
 # From Paul Eggert (2014-07-08):
 # That web page currently lists transitions for 2012/3 and 2013/4.
 # Assume the pattern instituted in 2012 will continue indefinitely.
+#
+# From Geoffrey D. Bennett (2021-09-20):
+# https://www.mcil.gov.ws/storage/2021/09/MCIL-Scan_20210920_120553.pdf
+# DST has been cancelled for this year.
 
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
 Rule	WS	2010	only	-	Sep	lastSun	0:00	1	-
 Rule	WS	2011	only	-	Apr	Sat>=1	4:00	0	-
 Rule	WS	2011	only	-	Sep	lastSat	3:00	1	-
-Rule	WS	2012	max	-	Apr	Sun>=1	4:00	0	-
-Rule	WS	2012	max	-	Sep	lastSun	3:00	1	-
+Rule	WS	2012	2021	-	Apr	Sun>=1	4:00	0	-
+Rule	WS	2012	2020	-	Sep	lastSun	3:00	1	-
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone Pacific/Apia	 12:33:04 -	LMT	1892 Jul  5
 			-11:26:56 -	LMT	1911
@@ -818,8 +864,8 @@ Rule	Tonga	2001	2002	-	Jan	lastSun	2:00	0	-
 Rule	Tonga	2016	only	-	Nov	Sun>=1	2:00	1:00	-
 Rule	Tonga	2017	only	-	Jan	Sun>=15	3:00	0	-
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone Pacific/Tongatapu	12:19:20 -	LMT	1901
-			12:20	-	+1220	1941
+Zone Pacific/Tongatapu	12:19:12 -	LMT	1945 Sep 10
+			12:20	-	+1220	1961
 			13:00	-	+13	1999
 			13:00	Tonga	+13/+14
 
@@ -1761,6 +1807,23 @@ Zone	Pacific/Wallis	12:15:20 -	LMT	1901
 # One source for this is page 202 of: Bartky IR. One Time Fits All:
 # The Campaigns for Global Uniformity (2007).
 
+# Kanton
+
+# From Paul Eggert (2021-05-27):
+# Kiribati's +13 timezone is represented by Kanton, its only populated
+# island.  (It was formerly spelled "Canton", but Gilbertese lacks "C".)
+# Kanton was settled on 1937-08-31 by two British radio operators
+# <https://history.state.gov/historicaldocuments/frus1937v02/d94>;
+# Americans came the next year and built an airfield, partly to
+# establish airline service and perhaps partly anticipating the
+# next war.  Aside from the war, the airfield was used by commercial
+# airlines until long-range jets became standard; although currently
+# for emergency use only, China says it is considering rebuilding the
+# airfield for high-end niche tourism.  Kanton has about two dozen
+# people, caretakers who rotate in from the rest of Kiribati in 2-5
+# year shifts, and who use some of the leftover structures
+# <http://pipa.neaq.org/2012/06/images-of-kanton-island.html>.
+
 # Kwajalein
 
 # From an AP article (1993-08-22):
@@ -2044,6 +2107,17 @@ Zone	Pacific/Wallis	12:15:20 -	LMT	1901
 
 # Tonga
 
+# From Paul Eggert (2021-03-04):
+# In 1943 "The standard time kept is 12 hrs. 19 min. 12 sec. fast
+# on Greenwich mean time." according to the Admiralty's Hydrographic
+# Dept., Pacific Islands Pilot, Vol. II, 7th ed., 1943, p 360.
+
+# From Michael Deckers (2021-03-03):
+# [Ian R Bartky: "One Time Fits All: The Campaigns for Global Uniformity".
+# Stanford University Press. 2007. p. 255]:
+# On 10 September 1945 Tonga adopted a standard time 12 hours,
+# 20 minutes in advance of Greenwich.
+
 # From Paul Eggert (1996-01-22):
 # Today's _Wall Street Journal_ (p 1) reports that "Tonga has been plotting
 # to sneak ahead of [New Zealanders] by introducing daylight-saving time."
@@ -2072,9 +2146,26 @@ Zone	Pacific/Wallis	12:15:20 -	LMT	1901
 # The Crown Prince, presented an unanswerable argument: "Remember that
 # on the World Day of Prayer, you would be the first people on Earth
 # to say your prayers in the morning."
-
-# From Paul Eggert (2006-03-22):
-# Shanks & Pottenger say the transition was on 1968-10-01; go with Mundell.
+#
+# From Tim Parenti (2021-09-13), per Paul Eggert (2006-03-22) and Michael
+# Deckers (2021-03-03):
+# Mundell places the transition from +12:20 to +13 in 1941, while Shanks &
+# Pottenger say the transition was on 1968-10-01.
+#
+# The Air Almanac published contemporaneous tables of standard times,
+# which listed +12:20 as of Nov 1960 and +13 as of Mar 1961:
+#   Nov 1960: https://books.google.com/books?id=bVgtWM6kPZUC&pg=SL1-PA19
+#   Mar 1961: https://books.google.com/books?id=W2nItAul4g0C&pg=SL1-PA19
+# (Thanks to P Chan for pointing us toward these sources.)
+# This agrees with Bartky, who writes that "since 1961 [Tonga's] official time
+# has been thirteen hours in advance of Greenwich time" (p. 202) and further
+# writes in an endnote that this was because "the legislation was amended" on
+# 1960-10-19. (p. 255)
+#
+# Without greater specificity, presume that Bartky and the Air Almanac point to
+# a 1961-01-01 transition, as Tāufaʻāhau Tupou IV was still Crown Prince in
+# 1961 and this still jives with the gist of Mundell's telling, and go with
+# this over Shanks & Pottenger.
 
 # From Eric Ulevik (1999-05-03):
 # Tonga's director of tourism, who is also secretary of the National Millennium

make/data/tzdata/backward

@@ -26,8 +26,10 @@
 # This file is in the public domain, so clarified as of
 # 2009-05-17 by Arthur David Olson.
 
-# This file provides links between current names for timezones
-# and their old names.  Many names changed in late 1993.
+# This file provides links from old or merged timezone names to current ones.
+# Many names changed in late 1993.  Several of these names are
+# also present in the file 'backzone', which has data important only
+# for pre-1970 timestamps and so is out of scope for tzdb proper.
 
 # Link	TARGET			LINK-NAME
 Link	Africa/Nairobi		Africa/Asmera
@@ -36,7 +38,7 @@ Link	America/Argentina/Catamarca	America/Argentina/ComodRivadavia
 Link	America/Adak		America/Atka
 Link	America/Argentina/Buenos_Aires	America/Buenos_Aires
 Link	America/Argentina/Catamarca	America/Catamarca
-Link	America/Atikokan	America/Coral_Harbour
+Link	America/Panama		America/Coral_Harbour
 Link	America/Argentina/Cordoba	America/Cordoba
 Link	America/Tijuana		America/Ensenada
 Link	America/Indiana/Indianapolis	America/Fort_Wayne
@@ -51,7 +53,7 @@ Link	America/Rio_Branco	America/Porto_Acre
 Link	America/Argentina/Cordoba	America/Rosario
 Link	America/Tijuana		America/Santa_Isabel
 Link	America/Denver		America/Shiprock
-Link	America/Port_of_Spain	America/Virgin
+Link	America/Puerto_Rico	America/Virgin
 Link	Pacific/Auckland	Antarctica/South_Pole
 Link	Asia/Ashgabat		Asia/Ashkhabad
 Link	Asia/Kolkata		Asia/Calcutta
@@ -126,6 +128,7 @@ Link	Pacific/Auckland	NZ
 Link	Pacific/Chatham		NZ-CHAT
 Link	America/Denver		Navajo
 Link	Asia/Shanghai		PRC
+Link	Pacific/Kanton		Pacific/Enderbury
 Link	Pacific/Honolulu	Pacific/Johnston
 Link	Pacific/Pohnpei		Pacific/Ponape
 Link	Pacific/Pago_Pago	Pacific/Samoa

make/data/tzdata/europe

@@ -91,7 +91,6 @@
 #        0:00       GMT  BST  BDST  Greenwich, British Summer
 #        0:00       GMT  IST        Greenwich, Irish Summer
 #        0:00       WET  WEST WEMT  Western Europe
-#        0:19:32.13 AMT* NST*       Amsterdam, Netherlands Summer (1835-1937)
 #        1:00       BST             British Standard (1968-1971)
 #        1:00       IST  GMT        Irish Standard (1968-) with winter DST
 #        1:00       CET  CEST CEMT  Central Europe
@@ -1823,6 +1822,10 @@ Zone	Europe/Rome	0:49:56 -	LMT	1866 Dec 12
 			1:00	Italy	CE%sT	1980
 			1:00	EU	CE%sT
 
+# Kosovo
+# See Europe/Belgrade.
+
+
 Link	Europe/Rome	Europe/Vatican
 Link	Europe/Rome	Europe/San_Marino
 
@@ -2173,6 +2176,10 @@ Zone	Europe/Monaco	0:29:32 -	LMT	1892 Jun  1
 # The data entries before 1945 are taken from
 # https://www.staff.science.uu.nl/~gent0113/wettijd/wettijd.htm
 
+# From Paul Eggert (2021-05-09):
+# I invented the abbreviations AMT for Amsterdam Mean Time and NST for
+# Netherlands Summer Time, used in the Netherlands from 1835 to 1937.
+
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
 Rule	Neth	1916	only	-	May	 1	0:00	1:00	NST	# Netherlands Summer Time
 Rule	Neth	1916	only	-	Oct	 1	0:00	0	AMT	# Amsterdam Mean Time
@@ -2399,12 +2406,10 @@ Rule	Port	1943	1945	-	Aug	Sat>=25	22:00s	1:00	S
 Rule	Port	1944	1945	-	Apr	Sat>=21	22:00s	2:00	M
 Rule	Port	1946	only	-	Apr	Sat>=1	23:00s	1:00	S
 Rule	Port	1946	only	-	Oct	Sat>=1	23:00s	0	-
-Rule	Port	1947	1949	-	Apr	Sun>=1	 2:00s	1:00	S
-Rule	Port	1947	1949	-	Oct	Sun>=1	 2:00s	0	-
-# Shanks & Pottenger say DST was observed in 1950; go with Whitman.
+# Whitman says DST was not observed in 1950; go with Shanks & Pottenger.
 # Whitman gives Oct lastSun for 1952 on; go with Shanks & Pottenger.
-Rule	Port	1951	1965	-	Apr	Sun>=1	 2:00s	1:00	S
-Rule	Port	1951	1965	-	Oct	Sun>=1	 2:00s	0	-
+Rule	Port	1947	1965	-	Apr	Sun>=1	 2:00s	1:00	S
+Rule	Port	1947	1965	-	Oct	Sun>=1	 2:00s	0	-
 Rule	Port	1977	only	-	Mar	27	 0:00s	1:00	S
 Rule	Port	1977	only	-	Sep	25	 0:00s	0	-
 Rule	Port	1978	1979	-	Apr	Sun>=1	 0:00s	1:00	S
@@ -3706,6 +3711,9 @@ Zone	Atlantic/Canary	-1:01:36 -	LMT	1922 Mar # Las Palmas de Gran C.
 #
 # Source: The newspaper "Dagens Nyheter", 1916-10-01, page 7 upper left.
 
+# An extra-special abbreviation style is SET for Swedish Time (svensk
+# normaltid) 1879-1899, 3° west of the Stockholm Observatory.
+
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
 Zone Europe/Stockholm	1:12:12 -	LMT	1879 Jan  1
 			1:00:14	-	SET	1900 Jan  1 # Swedish Time

make/data/tzdata/leapseconds

@@ -95,11 +95,11 @@ Leap	2016	Dec	31	23:59:60	+	S
 # Any additional leap seconds will come after this.
 # This Expires line is commented out for now,
 # so that pre-2020a zic implementations do not reject this file.
-#Expires 2021	Dec	28	00:00:00
+#Expires 2022	Jun	28	00:00:00
 
 # POSIX timestamps for the data in this file:
 #updated 1467936000 (2016-07-08 00:00:00 UTC)
-#expires 1640649600 (2021-12-28 00:00:00 UTC)
+#expires 1656374400 (2022-06-28 00:00:00 UTC)
 
-#	Updated through IERS Bulletin C61
-#	File expires on:  28 December 2021
+#	Updated through IERS Bulletin C62
+#	File expires on:  28 June 2022

make/data/tzdata/northamerica

@@ -752,7 +752,11 @@ Zone America/Adak	 12:13:22 -	LMT	1867 Oct 19 12:44:35
 			-11:00	US	B%sT	1983 Oct 30  2:00
 			-10:00	US	AH%sT	1983 Nov 30
 			-10:00	US	H%sT
-# The following switches don't quite make our 1970 cutoff.
+# The following switches don't make our 1970 cutoff.
+#
+# Kiska observed Tokyo date and time during Japanese occupation from
+# 1942-06-06 to 1943-07-29, and similarly for Attu from 1942-06-07 to
+# 1943-05-29 (all dates American).  Both islands are now uninhabited.
 #
 # Shanks writes that part of southwest Alaska (e.g. Aniak)
 # switched from -11:00 to -10:00 on 1968-09-22 at 02:00,
@@ -848,6 +852,8 @@ Zone America/Phoenix	-7:28:18 -	LMT	1883 Nov 18 11:31:42
 			-7:00	-	MST	1967
 			-7:00	US	M%sT	1968 Mar 21
 			-7:00	-	MST
+Link America/Phoenix America/Creston
+
 # From Arthur David Olson (1988-02-13):
 # A writer from the Inter Tribal Council of Arizona, Inc.,
 # notes in private correspondence dated 1987-12-28 that "Presently, only the
@@ -1616,24 +1622,7 @@ Zone America/Moncton	-4:19:08 -	LMT	1883 Dec  9
 # From Paul Eggert (2020-01-10):
 # See America/Toronto for most of Quebec, including Montreal.
 # See America/Halifax for the Îles de la Madeleine and the Listuguj reserve.
-#
-# Matthews and Vincent (1998) also write that Quebec east of the -63
-# meridian is supposed to observe AST, but residents as far east as
-# Natashquan use EST/EDT, and residents east of Natashquan use AST.
-# The Quebec department of justice writes in
-# "The situation in Minganie and Basse-Côte-Nord"
-# https://www.justice.gouv.qc.ca/en/department/ministre/functions-and-responsabilities/legal-time-in-quebec/the-situation-in-minganie-and-basse-cote-nord/
-# that the coastal strip from just east of Natashquan to Blanc-Sablon
-# observes Atlantic standard time all year round.
-# This common practice was codified into law as of 2007; see Legal Time Act,
-# CQLR c T-5.1 <http://legisquebec.gouv.qc.ca/en/ShowDoc/cs/T-5.1>.
-# For lack of better info, guess this practice began around 1970, contra to
-# Shanks & Pottenger who have this region observing AST/ADT.
-
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone America/Blanc-Sablon -3:48:28 -	LMT	1884
-			-4:00	Canada	A%sT	1970
-			-4:00	-	AST
+# See America/Puerto_Rico for east of Natashquan.
 
 # Ontario
 
@@ -1672,54 +1661,6 @@ Zone America/Blanc-Sablon -3:48:28 -	LMT	1884
 # time became a comic failure in Orillia. Toronto Star 2017-07-08.
 # https://www.thestar.com/news/insight/2017/07/08/bold-attempt-at-daylight-saving-time-became-a-comic-failure-in-orillia.html
 
-# From Paul Eggert (1997-10-17):
-# Mark Brader writes that an article in the 1997-10-14 Toronto Star
-# says that Atikokan, Ontario currently does not observe DST,
-# but will vote on 11-10 whether to use EST/EDT.
-# He also writes that the Ontario Time Act (1990, Chapter T.9)
-# http://www.gov.on.ca/MBS/english/publications/statregs/conttext.html
-# says that Ontario east of 90W uses EST/EDT, and west of 90W uses CST/CDT.
-# Officially Atikokan is therefore on CST/CDT, and most likely this report
-# concerns a non-official time observed as a matter of local practice.
-#
-# From Paul Eggert (2000-10-02):
-# Matthews and Vincent (1998) write that Atikokan, Pickle Lake, and
-# New Osnaburgh observe CST all year, that Big Trout Lake observes
-# CST/CDT, and that Upsala and Shebandowan observe EST/EDT, all in
-# violation of the official Ontario rules.
-#
-# From Paul Eggert (2006-07-09):
-# Chris Walton (2006-07-06) mentioned an article by Stephanie MacLellan in the
-# 2005-07-21 Chronicle-Journal, which said:
-#
-#	The clocks in Atikokan stay set on standard time year-round.
-#	This means they spend about half the time on central time and
-#	the other half on eastern time.
-#
-#	For the most part, the system works, Mayor Dennis Brown said.
-#
-#	"The majority of businesses in Atikokan deal more with Eastern
-#	Canada, but there are some that deal with Western Canada," he
-#	said.  "I don't see any changes happening here."
-#
-# Walton also writes "Supposedly Pickle Lake and Mishkeegogamang
-# [New Osnaburgh] follow the same practice."
-
-# From Garry McKinnon (2006-07-14) via Chris Walton:
-# I chatted with a member of my board who has an outstanding memory
-# and a long history in Atikokan (and in the telecom industry) and he
-# can say for certain that Atikokan has been practicing the current
-# time keeping since 1952, at least.
-
-# From Paul Eggert (2006-07-17):
-# Shanks & Pottenger say that Atikokan has agreed with Rainy River
-# ever since standard time was introduced, but the information from
-# McKinnon sounds more authoritative.  For now, assume that Atikokan
-# switched to EST immediately after WWII era daylight saving time
-# ended.  This matches the old (less-populous) America/Coral_Harbour
-# entry since our cutoff date of 1970, so we can move
-# America/Coral_Harbour to the 'backward' file.
-
 # From Mark Brader (2010-03-06):
 #
 # Currently the database has:
@@ -1850,6 +1791,7 @@ Zone America/Toronto	-5:17:32 -	LMT	1895
 			-5:00	Canada	E%sT	1946
 			-5:00	Toronto	E%sT	1974
 			-5:00	Canada	E%sT
+Link America/Toronto America/Nassau
 Zone America/Thunder_Bay -5:57:00 -	LMT	1895
 			-6:00	-	CST	1910
 			-5:00	-	EST	1942
@@ -1865,11 +1807,7 @@ Zone America/Rainy_River -6:18:16 -	LMT	1895
 			-6:00	Canada	C%sT	1940 Sep 29
 			-6:00	1:00	CDT	1942 Feb  9  2:00s
 			-6:00	Canada	C%sT
-Zone America/Atikokan	-6:06:28 -	LMT	1895
-			-6:00	Canada	C%sT	1940 Sep 29
-			-6:00	1:00	CDT	1942 Feb  9  2:00s
-			-6:00	Canada	C%sT	1945 Sep 30  2:00
-			-5:00	-	EST
+# For Atikokan see America/Panama.
 
 
 # Manitoba
@@ -2060,60 +1998,6 @@ Zone America/Edmonton	-7:33:52 -	LMT	1906 Sep
 # Shanks & Pottenger write that since 1970 most of this region has
 # been like Vancouver.
 # Dawson Creek uses MST.  Much of east BC is like Edmonton.
-# Matthews and Vincent (1998) write that Creston is like Dawson Creek.
-
-# It seems though that (re: Creston) is not entirely correct:
-
-# From Chris Walton (2011-12-01):
-# There are two areas within the Canadian province of British Columbia
-# that do not currently observe daylight saving:
-# a) The Creston Valley (includes the town of Creston and surrounding area)
-# b) The eastern half of the Peace River Regional District
-# (includes the cities of Dawson Creek and Fort St. John)
-
-# Earlier this year I stumbled across a detailed article about the time
-# keeping history of Creston; it was written by Tammy Hardwick who is the
-# manager of the Creston & District Museum. The article was written in May 2009.
-# http://www.ilovecreston.com/?p=articles&t=spec&ar=260
-# According to the article, Creston has not changed its clocks since June 1918.
-# i.e. Creston has been stuck on UT-7 for 93 years.
-# Dawson Creek, on the other hand, changed its clocks as recently as April 1972.
-
-# Unfortunately the exact date for the time change in June 1918 remains
-# unknown and will be difficult to ascertain.  I e-mailed Tammy a few months
-# ago to ask if Sunday June 2 was a reasonable guess.  She said it was just
-# as plausible as any other date (in June).  She also said that after writing
-# the article she had discovered another time change in 1916; this is the
-# subject of another article which she wrote in October 2010.
-# http://www.creston.museum.bc.ca/index.php?module=comments&uop=view_comment&cm+id=56
-
-# Here is a summary of the three clock change events in Creston's history:
-# 1. 1884 or 1885: adoption of Mountain Standard Time (GMT-7)
-# Exact date unknown
-# 2. Oct 1916: switch to Pacific Standard Time (GMT-8)
-# Exact date in October unknown; Sunday October 1 is a reasonable guess.
-# 3. June 1918: switch to Pacific Daylight Time (GMT-7)
-# Exact date in June unknown; Sunday June 2 is a reasonable guess.
-# note 1:
-# On Oct 27/1918 when daylight saving ended in the rest of Canada,
-# Creston did not change its clocks.
-# note 2:
-# During WWII when the Federal Government legislated a mandatory clock change,
-# Creston did not oblige.
-# note 3:
-# There is no guarantee that Creston will remain on Mountain Standard Time
-# (UTC-7) forever.
-# The subject was debated at least once this year by the town Council.
-# http://www.bclocalnews.com/kootenay_rockies/crestonvalleyadvance/news/116760809.html
-
-# During a period WWII, summer time (Daylight saying) was mandatory in Canada.
-# In Creston, that was handled by shifting the area to PST (-8:00) then applying
-# summer time to cause the offset to be -7:00, the same as it had been before
-# the change.  It can be argued that the timezone abbreviation during this
-# period should be PDT rather than MST, but that doesn't seem important enough
-# (to anyone) to further complicate the rules.
-
-# The transition dates (and times) are guesses.
 
 # From Matt Johnson (2015-09-21):
 # Fort Nelson, BC, Canada will cancel DST this year.  So while previously they
@@ -2167,10 +2051,7 @@ Zone America/Fort_Nelson	-8:10:47 -	LMT	1884
 			-8:00	Vanc	P%sT	1987
 			-8:00	Canada	P%sT	2015 Mar  8  2:00
 			-7:00	-	MST
-Zone America/Creston	-7:46:04 -	LMT	1884
-			-7:00	-	MST	1916 Oct 1
-			-8:00	-	PST	1918 Jun 2
-			-7:00	-	MST
+# For Creston see America/Phoenix.
 
 # Northwest Territories, Nunavut, Yukon
 
@@ -2952,64 +2833,61 @@ Zone America/Tijuana	-7:48:04 -	LMT	1922 Jan  1  0:11:56
 
 # Anguilla
 # Antigua and Barbuda
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
 
-# Bahamas
-#
-# For 1899 Milne gives -5:09:29.5; round that.
-#
-# From P Chan (2020-11-27, corrected on 2020-12-02):
-# There were two periods of DST observed in 1942-1945: 1942-05-01
-# midnight to 1944-12-31 midnight and 1945-02-01 to 1945-10-17 midnight.
-# "midnight" should mean 24:00 from the context.
-#
-# War Time Order 1942 [1942-05-01] and War Time (No. 2) Order 1942  [1942-09-29]
-# Appendix to the Statutes of 7 George VI. and the Year 1942. p 34, 43
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA3-PA34
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA3-PA43
-#
-# War Time Order 1943 [1943-03-31] and War Time Order 1944 [1943-12-29]
-# Appendix to the Statutes of 8 George VI. and the Year 1943. p 9-10, 28-29
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA4-PA9
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA4-PA28
-#
-# War Time Order 1945 [1945-01-31] and the Order which revoke War Time Order
-# 1945 [1945-10-16] Appendix to the Statutes of 9 George VI. and the Year
-# 1945. p 160, 247-248
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA6-PA160
-# https://books.google.com/books?id=5rlNAQAAIAAJ&pg=RA6-PA247
-#
-# From Sue Williams (2006-12-07):
-# The Bahamas announced about a month ago that they plan to change their DST
-# rules to sync with the U.S. starting in 2007....
-# http://www.jonesbahamas.com/?c=45&a=10412
+# The Bahamas
+# See America/Toronto.
 
-# Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
-Rule	Bahamas	1942	only	-	May	 1	24:00	1:00	W
-Rule	Bahamas	1944	only	-	Dec	31	24:00	0	S
-Rule	Bahamas	1945	only	-	Feb	 1	0:00	1:00	W
-Rule	Bahamas	1945	only	-	Aug	14	23:00u	1:00	P # Peace
-Rule	Bahamas	1945	only	-	Oct	17	24:00	0	S
-Rule	Bahamas	1964	1975	-	Oct	lastSun	2:00	0	S
-Rule	Bahamas	1964	1975	-	Apr	lastSun	2:00	1:00	D
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone	America/Nassau	-5:09:30 -	LMT	1912 Mar 2
-			-5:00	Bahamas	E%sT	1976
-			-5:00	US	E%sT
 
 # Barbados
 
 # For 1899 Milne gives -3:58:29.2; round that.
 
+# From P Chan (2020-12-09 and 2020-12-11):
+# Standard time of GMT-4 was adopted in 1911.
+# Definition of Time Act, 1911 (1911-7) [1911-08-28]
+# 1912, Laws of Barbados (5 v.), OCLC Number: 919801291, Vol. 4, Image No. 522
+# 1944, Laws of Barbados (5 v.), OCLC Number: 84548697, Vol. 4, Image No. 122
+# http://llmc.com/browse.aspx?type=2&coll=85&div=297
+#
+# DST was observed in 1942-44.
+# Defence (Daylight Saving) Regulations, 1942, 1942-04-13
+# Defence (Daylight Saving) (Repeal) Regulations, 1942, 1942-08-22
+# Defence (Daylight Saving) Regulations, 1943, 1943-04-16
+# Defence (Daylight Saving) (Repeal) Regulations, 1943, 1943-09-01
+# Defence (Daylight Saving) Regulations, 1944, 1944-03-21
+# [Defence (Daylight Saving) (Amendment) Regulations 1944, 1944-03-28]
+# Defence (Daylight Saving) (Repeal) Regulations, 1944, 1944-08-30
+#
+# 1914-, Subsidiary Legis., Annual Vols. OCLC Number: 226290591
+# 1942: Image Nos. 527-528, 555-556
+# 1943: Image Nos. 178-179, 198
+# 1944: Image Nos. 113-115, 129
+# http://llmc.com/titledescfull.aspx?type=2&coll=85&div=297&set=98437
+#
+# From Tim Parenti (2021-02-20):
+# The transitions below are derived from P Chan's sources, except that the 1977
+# through 1980 transitions are from Shanks & Pottenger since we have no better
+# data there.  Of particular note, the 1944 DST regulation only advanced the
+# time to "exactly three and a half hours later than Greenwich mean time", as
+# opposed to "three hours" in the 1942 and 1943 regulations.
+
 # Rule	NAME	FROM	TO	-	IN	ON	AT	SAVE	LETTER/S
+Rule	Barb	1942	only	-	Apr	19	5:00u	1:00	D
+Rule	Barb	1942	only	-	Aug	31	6:00u	0	S
+Rule	Barb	1943	only	-	May	 2	5:00u	1:00	D
+Rule	Barb	1943	only	-	Sep	 5	6:00u	0	S
+Rule	Barb	1944	only	-	Apr	10	5:00u	0:30	-
+Rule	Barb	1944	only	-	Sep	10	6:00u	0	S
 Rule	Barb	1977	only	-	Jun	12	2:00	1:00	D
 Rule	Barb	1977	1978	-	Oct	Sun>=1	2:00	0	S
 Rule	Barb	1978	1980	-	Apr	Sun>=15	2:00	1:00	D
 Rule	Barb	1979	only	-	Sep	30	2:00	0	S
 Rule	Barb	1980	only	-	Sep	25	2:00	0	S
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone America/Barbados	-3:58:29 -	LMT	1924 # Bridgetown
-			-3:58:29 -	BMT	1932 # Bridgetown Mean Time
+Zone America/Barbados	-3:58:29 -	LMT	1911 Aug 28 # Bridgetown
+			-4:00	Barb	A%sT	1944
+			-4:00	Barb	AST/-0330 1945
 			-4:00	Barb	A%sT
 
 # Belize
@@ -3171,6 +3049,9 @@ Zone Atlantic/Bermuda	-4:19:18 -	LMT	1890	# Hamilton
 			-4:00	Canada	A%sT	1976
 			-4:00	US	A%sT
 
+# Caribbean Netherlands
+# See America/Puerto_Rico.
+
 # Cayman Is
 # See America/Panama.
 
@@ -3399,7 +3280,7 @@ Zone	America/Havana	-5:29:28 -	LMT	1890
 			-5:00	Cuba	C%sT
 
 # Dominica
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
 
 # Dominican Republic
 
@@ -3451,7 +3332,7 @@ Zone America/El_Salvador -5:56:48 -	LMT	1921 # San Salvador
 # Guadeloupe
 # St Barthélemy
 # St Martin (French part)
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
 
 # Guatemala
 #
@@ -3638,7 +3519,7 @@ Zone America/Martinique	-4:04:20 -      LMT	1890        # Fort-de-France
 			-4:00	-	AST
 
 # Montserrat
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
 
 # Nicaragua
 #
@@ -3710,6 +3591,7 @@ Zone	America/Managua	-5:45:08 -	LMT	1890
 Zone	America/Panama	-5:18:08 -	LMT	1890
 			-5:19:36 -	CMT	1908 Apr 22 # Colón Mean Time
 			-5:00	-	EST
+Link America/Panama America/Atikokan
 Link America/Panama America/Cayman
 
 # Puerto Rico
@@ -3719,10 +3601,29 @@ Zone America/Puerto_Rico -4:24:25 -	LMT	1899 Mar 28 12:00 # San Juan
 			-4:00	-	AST	1942 May  3
 			-4:00	US	A%sT	1946
 			-4:00	-	AST
+Link America/Puerto_Rico America/Anguilla
+Link America/Puerto_Rico America/Antigua
+Link America/Puerto_Rico America/Aruba
+Link America/Puerto_Rico America/Curacao
+Link America/Puerto_Rico America/Blanc-Sablon	# Quebec (Lower North Shore)
+Link America/Puerto_Rico America/Dominica
+Link America/Puerto_Rico America/Grenada
+Link America/Puerto_Rico America/Guadeloupe
+Link America/Puerto_Rico America/Kralendijk	# Caribbean Netherlands
+Link America/Puerto_Rico America/Lower_Princes	# Sint Maarten
+Link America/Puerto_Rico America/Marigot	# St Martin (French part)
+Link America/Puerto_Rico America/Montserrat
+Link America/Puerto_Rico America/Port_of_Spain	# Trinidad & Tobago
+Link America/Puerto_Rico America/St_Barthelemy	# St Barthélemy
+Link America/Puerto_Rico America/St_Kitts	# St Kitts & Nevis
+Link America/Puerto_Rico America/St_Lucia
+Link America/Puerto_Rico America/St_Thomas	# Virgin Islands (US)
+Link America/Puerto_Rico America/St_Vincent
+Link America/Puerto_Rico America/Tortola	# Virgin Islands (UK)
 
 # St Kitts-Nevis
 # St Lucia
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
 
 # St Pierre and Miquelon
 # There are too many St Pierres elsewhere, so we'll use 'Miquelon'.
@@ -3733,7 +3634,10 @@ Zone America/Miquelon	-3:44:40 -	LMT	1911 May 15 # St Pierre
 			-3:00	Canada	-03/-02
 
 # St Vincent and the Grenadines
-# See America/Port_of_Spain.
+# See America/Puerto_Rico.
+
+# Sint Maarten
+# See America/Puerto_Rico.
 
 # Turks and Caicos
 #
@@ -3804,8 +3708,8 @@ Zone America/Grand_Turk	-4:44:32 -	LMT	1890
 			-5:00	US	E%sT
 
 # British Virgin Is
-# Virgin Is
-# See America/Port_of_Spain.
+# US Virgin Is
+# See America/Puerto_Rico.
 
 
 # Local Variables:

make/data/tzdata/southamerica

@@ -597,7 +597,7 @@ Zone America/Argentina/Ushuaia -4:33:12 - LMT	1894 Oct 31
 			-3:00	-	-03
 
 # Aruba
-Link America/Curacao America/Aruba
+# See America/Puerto_Rico.
 
 # Bolivia
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
@@ -1392,35 +1392,14 @@ Zone	America/Bogota	-4:56:16 -	LMT	1884 Mar 13
 # no information; probably like America/Bogota
 
 # Curaçao
-
-# Milne gives 4:35:46.9 for Curaçao mean time; round to nearest.
+# See America/Puerto_Rico.
 #
-# From Paul Eggert (2006-03-22):
-# Shanks & Pottenger say that The Bottom and Philipsburg have been at
-# -4:00 since standard time was introduced on 1912-03-02; and that
-# Kralendijk and Rincon used Kralendijk Mean Time (-4:33:08) from
-# 1912-02-02 to 1965-01-01.  The former is dubious, since S&P also say
-# Saba Island has been like Curaçao.
-# This all predates our 1970 cutoff, though.
-#
-# By July 2007 Curaçao and St Maarten are planned to become
-# associated states within the Netherlands, much like Aruba;
-# Bonaire, Saba and St Eustatius would become directly part of the
-# Netherlands as Kingdom Islands.  This won't affect their time zones
-# though, as far as we know.
-#
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone	America/Curacao	-4:35:47 -	LMT	1912 Feb 12 # Willemstad
-			-4:30	-	-0430	1965
-			-4:00	-	AST
-
 # From Arthur David Olson (2011-06-15):
 # use links for places with new iso3166 codes.
 # The name "Lower Prince's Quarter" is both longer than fourteen characters
-# and contains an apostrophe; use "Lower_Princes" below.
-
-Link	America/Curacao	America/Lower_Princes	# Sint Maarten
-Link	America/Curacao	America/Kralendijk	# Caribbean Netherlands
+# and contains an apostrophe; use "Lower_Princes"....
+# From Paul Eggert (2021-09-29):
+# These backward-compatibility links now are in the 'northamerica' file.
 
 # Ecuador
 #
@@ -1563,11 +1542,40 @@ Zone America/Cayenne	-3:29:20 -	LMT	1911 Jul
 			-3:00	-	-03
 
 # Guyana
+
+# From P Chan (2020-11-27):
+# https://books.google.com/books?id=5-5CAQAAMAAJ&pg=SA1-PA547
+# The Official Gazette of British Guiana. (New Series.) Vol. XL. July to
+# December, 1915, p 1547, lists as several notes:
+# "Local Mean Time 3 hours 52 mins. 39 secs. slow of Greenwich Mean Time
+# (Georgetown.) From 1st August, 1911, British Guiana Standard Mean Time 4
+# hours slow of Greenwich Mean Time, by notice in Official Gazette on 1st July,
+# 1911.  From 1st March, 1915, British Guiana Standard Mean Time 3 hours 45
+# mins. 0 secs. slow of Greenwich Mean Time, by notice in Official Gazette on
+# 23rd January, 1915."
+#
+# https://parliament.gov.gy/documents/acts/10923-act_no._27_of_1975_-_interpretation_and_general_clauses_(amendment)_act_1975.pdf
+# Interpretation and general clauses (Amendment) Act 1975 (Act No. 27 of 1975)
+# [dated 1975-07-31]
+# "This Act...shall come into operation on 1st August, 1975."
+# "...where any expression of time occurs...the time referred to shall signify
+# the standard time of Guyana which shall be three hours behind Greenwich Mean
+# Time."
+#
+# Circular No. 10/1992 dated 1992-03-20
+# https://dps.gov.gy/wp-content/uploads/2018/12/1992-03-20-Circular-010.pdf
+# "...cabinet has decided that with effect from Sunday 29th March, 1992, Guyana
+# Standard Time would be re-established at 01:00 hours by adjusting the hands
+# of the clock back to 24:00 hours."
+# Legislated in the Interpretation and general clauses (Amendment) Act 1992
+# (Act No. 6 of 1992) [passed 1992-03-27, published 1992-04-18]
+# https://parliament.gov.gy/documents/acts/5885-6_of_1992_interpretation_and_general_clauses_(amendment)_act_1992.pdf
+
 # Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone	America/Guyana	-3:52:40 -	LMT	1915 Mar    # Georgetown
-			-3:45	-	-0345	1975 Jul 31
-			-3:00	-	-03	1991
-# IATA SSIM (1996-06) says -4:00.  Assume a 1991 switch.
+Zone	America/Guyana	-3:52:39 -	LMT	1911 Aug  1 # Georgetown
+			-4:00	-	-04	1915 Mar  1
+			-3:45	-	-0345	1975 Aug  1
+			-3:00	-	-03	1992 Mar 29  1:00
 			-4:00	-	-04
 
 # Paraguay
@@ -1708,24 +1716,7 @@ Zone America/Paramaribo	-3:40:40 -	LMT	1911
 			-3:00	-	-03
 
 # Trinidad and Tobago
-# Zone	NAME		STDOFF	RULES	FORMAT	[UNTIL]
-Zone America/Port_of_Spain -4:06:04 -	LMT	1912 Mar 2
-			-4:00	-	AST
-
-# These all agree with Trinidad and Tobago since 1970.
-Link America/Port_of_Spain America/Anguilla
-Link America/Port_of_Spain America/Antigua
-Link America/Port_of_Spain America/Dominica
-Link America/Port_of_Spain America/Grenada
-Link America/Port_of_Spain America/Guadeloupe
-Link America/Port_of_Spain America/Marigot	# St Martin (French part)
-Link America/Port_of_Spain America/Montserrat
-Link America/Port_of_Spain America/St_Barthelemy # St Barthélemy
-Link America/Port_of_Spain America/St_Kitts	# St Kitts & Nevis
-Link America/Port_of_Spain America/St_Lucia
-Link America/Port_of_Spain America/St_Thomas	# Virgin Islands (US)
-Link America/Port_of_Spain America/St_Vincent
-Link America/Port_of_Spain America/Tortola	# Virgin Islands (UK)
+# See America/Puerto_Rico.
 
 # Uruguay
 # From Paul Eggert (1993-11-18):

make/data/tzdata/zone.tab

@@ -26,7 +26,7 @@
 # This file is in the public domain, so clarified as of
 # 2009-05-17 by Arthur David Olson.
 #
-# From Paul Eggert (2018-06-27):
+# From Paul Eggert (2021-09-20):
 # This file is intended as a backward-compatibility aid for older programs.
 # New programs should use zone1970.tab.  This file is like zone1970.tab (see
 # zone1970.tab's comments), but with the following additional restrictions:
@@ -39,6 +39,9 @@
 # clocks have agreed since 1970; this is a narrower definition than
 # that of zone1970.tab.
 #
+# Unlike zone1970.tab, a row's third column can be a Link from
+# 'backward' instead of a Zone.
+#
 # This table is intended as an aid for users, to help them select timezones
 # appropriate for their practical needs.  It is not intended to take or
 # endorse any position on legal or territorial claims.
@@ -251,7 +254,7 @@ KE	-0117+03649	Africa/Nairobi
 KG	+4254+07436	Asia/Bishkek
 KH	+1133+10455	Asia/Phnom_Penh
 KI	+0125+17300	Pacific/Tarawa	Gilbert Islands
-KI	-0308-17105	Pacific/Enderbury	Phoenix Islands
+KI	-0247-17143	Pacific/Kanton	Phoenix Islands
 KI	+0152-15720	Pacific/Kiritimati	Line Islands
 KM	-1141+04316	Indian/Comoro
 KN	+1718-06243	America/St_Kitts
@@ -414,7 +417,7 @@ TK	-0922-17114	Pacific/Fakaofo
 TL	-0833+12535	Asia/Dili
 TM	+3757+05823	Asia/Ashgabat
 TN	+3648+01011	Africa/Tunis
-TO	-2110-17510	Pacific/Tongatapu
+TO	-210800-1751200	Pacific/Tongatapu
 TR	+4101+02858	Europe/Istanbul
 TT	+1039-06131	America/Port_of_Spain
 TV	-0831+17913	Pacific/Funafuti

make/hotspot/HotspotCommon.gmk

@@ -34,13 +34,7 @@ JVM_SUPPORT_DIR := $(JVM_VARIANT_OUTPUTDIR)/support
 DTRACE_SUPPORT_DIR := $(JVM_SUPPORT_DIR)/dtrace
 
 LIB_OUTPUTDIR := $(call FindLibDirForModule, java.base)
-ifneq ($(filter client minimal, $(JVM_VARIANT)), )
-  JVM_VARIANT_SUBDIR := $(JVM_VARIANT)
-else
-  # Use 'server' as default target directory name for all other variants.
-  JVM_VARIANT_SUBDIR := server
-endif
-JVM_LIB_OUTPUTDIR := $(LIB_OUTPUTDIR)/$(JVM_VARIANT_SUBDIR)
+JVM_LIB_OUTPUTDIR := $(LIB_OUTPUTDIR)/$(JVM_VARIANT)
 
 ################################################################################
 

make/hotspot/gensrc/GensrcAdlc.gmk

@@ -155,6 +155,7 @@ ifeq ($(call check-jvm-feature, compiler2), true)
   ifeq ($(call check-jvm-feature, zgc), true)
     AD_SRC_FILES += $(call uniq, $(wildcard $(foreach d, $(AD_SRC_ROOTS), \
         $d/cpu/$(HOTSPOT_TARGET_CPU_ARCH)/gc/z/z_$(HOTSPOT_TARGET_CPU).ad \
+        $d/cpu/$(HOTSPOT_TARGET_CPU_ARCH)/gc/z/z_$(HOTSPOT_TARGET_CPU_ARCH).ad \
       )))
   endif
 

make/hotspot/lib/CompileGtest.gmk

@@ -1,5 +1,5 @@
 #
-# Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved.
+# Copyright (c) 2016, 2021, Oracle and/or its affiliates. All rights reserved.
 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 #
 # This code is free software; you can redistribute it and/or modify it
@@ -101,7 +101,7 @@ $(eval $(call SetupJdkLibrary, BUILD_GTEST_LIBJVM, \
     CFLAGS_windows := -EHsc, \
     CFLAGS_macosx := -DGTEST_OS_MAC=1, \
     DISABLED_WARNINGS_gcc := $(DISABLED_WARNINGS_gcc) \
-        undef, \
+        undef stringop-overflow, \
     DISABLED_WARNINGS_clang := $(DISABLED_WARNINGS_clang) \
         undef switch format-nonliteral tautological-undefined-compare \
         self-assign-overloaded, \

make/hotspot/lib/JvmFeatures.gmk

@@ -58,6 +58,10 @@ else
   JVM_EXCLUDE_PATTERNS += /zero/
 endif
 
+ifeq ($(JVM_VARIANT), core)
+  JVM_CFLAGS_FEATURES += -DVMTYPE=\"Core\"
+endif
+
 ifeq ($(JVM_VARIANT), custom)
   JVM_CFLAGS_FEATURES += -DVMTYPE=\"Custom\"
 endif

make/jdk/src/classes/build/tools/generatelsrequivmaps/EquivMapsGenerator.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2012, 2021, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -27,7 +27,6 @@ package build.tools.generatelsrequivmaps;
 
 import java.io.BufferedWriter;
 import java.io.IOException;
-import java.nio.charset.Charset;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 import java.time.ZoneId;
@@ -79,8 +78,7 @@ public class EquivMapsGenerator {
         String preferred = null;
         String prefix = null;
 
-        for (String line : Files.readAllLines(Paths.get(filename),
-                                              Charset.forName("UTF-8"))) {
+        for (String line : Files.readAllLines(Paths.get(filename))) {
             line = line.toLowerCase(Locale.ROOT);
             int index = line.indexOf(' ') + 1;
             if (line.startsWith("file-date:")) {

make/jdk/src/classes/build/tools/jigsaw/GenGraphs.java

@@ -36,6 +36,7 @@ import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
@@ -115,102 +116,112 @@ public class GenGraphs {
     /**
      * Custom dot file attributes.
      */
-    static class ModuleGraphAttributes implements ModuleDotGraph.Attributes {
-        static Map<String, String> DEFAULT_ATTRIBUTES = Map.of(
-            "ranksep", "0.6",
-            "fontsize", "12",
-            "fontcolor", BLACK,
-            "fontname", "DejaVuSans",
-            "arrowsize", "1",
-            "arrowwidth", "2",
-            "arrowcolor", DARK_GRAY,
-            // custom
-            "requiresMandatedColor", LIGHT_GRAY,
-            "javaSubgraphColor", ORANGE,
-            "jdkSubgraphColor", BLUE
-        );
-
-        final Map<String, Integer> weights = new HashMap<>();
-        final List<Set<String>> ranks = new ArrayList<>();
-        final Map<String, String> attrs;
-        ModuleGraphAttributes(Map<String, String> attrs) {
-            int h = 1000;
-            weight("java.se", "java.sql.rowset", h * 10);
-            weight("java.sql.rowset", "java.sql", h * 10);
-            weight("java.sql", "java.xml", h * 10);
-            weight("java.xml", "java.base", h * 10);
-
-            ranks.add(Set.of("java.logging", "java.scripting", "java.xml"));
-            ranks.add(Set.of("java.sql"));
-            ranks.add(Set.of("java.transaction.xa"));
-            ranks.add(Set.of("java.compiler", "java.instrument"));
-            ranks.add(Set.of("java.desktop", "java.management"));
-
-            this.attrs = attrs;
-        }
+    static class ModuleGraphAttributes extends ModuleDotGraph.DotGraphAttributes {
+        final Properties attrs;
+        final Map<String, Integer> weights;
 
         ModuleGraphAttributes() {
-            this(DEFAULT_ATTRIBUTES);
-        }
+            this(new Properties());
+        };
         ModuleGraphAttributes(Properties props) {
-            this(toAttributes(props));
+            this.attrs = props;
+            this.weights = initWeights(props);
+        }
+
+        @Override
+        public double nodeSep() {
+            String v = attrs.getProperty("nodesep");
+            return v != null ? Double.valueOf(v) : super.nodeSep();
         }
 
         @Override
         public double rankSep() {
-            return Double.valueOf(attrs.get("ranksep"));
+            String v = attrs.getProperty("ranksep");
+            return v != null ? Double.valueOf(v) : super.rankSep();
         }
 
         @Override
         public int fontSize() {
-            return Integer.valueOf(attrs.get("fontsize"));
+            String v = attrs.getProperty("fontsize");
+            return v != null ? Integer.valueOf(v) : super.fontSize();
         }
 
         @Override
         public String fontName() {
-            return attrs.get("fontname");
+            String v = attrs.getProperty("fontname");
+            return v != null ? v : super.fontName();
         }
 
         @Override
         public String fontColor() {
-            return attrs.get("fontcolor");
+            String v = attrs.getProperty("fontcolor");
+            return v != null ? v : super.fontColor();
         }
 
         @Override
         public int arrowSize() {
-            return Integer.valueOf(attrs.get("arrowsize"));
+            String v = attrs.getProperty("arrowsize");
+            return v != null ? Integer.valueOf(v) : super.arrowSize();
         }
 
         @Override
         public int arrowWidth() {
-            return Integer.valueOf(attrs.get("arrowwidth"));
+            String v = attrs.getProperty("arrowwidth");
+            return v != null ? Integer.valueOf(v) : super.arrowWidth();
         }
 
         @Override
         public String arrowColor() {
-            return attrs.get("arrowcolor");
+            String v = attrs.getProperty("arrowcolor");
+            return v != null ? v : super.arrowColor();
         }
 
         @Override
         public List<Set<String>> ranks() {
-            return ranks;
+            return attrs.stringPropertyNames().stream()
+                        .filter(k -> k.startsWith("ranks."))
+                        .sorted()
+                        .map(k -> Arrays.stream(attrs.getProperty(k).split(","))
+                                        .collect(Collectors.toSet()))
+                        .toList();
         }
 
         @Override
         public String requiresMandatedColor() {
-            return attrs.get("requiresMandatedColor");
+            String v = attrs.getProperty("requiresMandatedColor");
+            return v != null ? v : super.requiresMandatedColor();
         }
 
         @Override
         public String javaSubgraphColor() {
-            return attrs.get("javaSubgraphColor");
+            String v = attrs.getProperty("javaSubgraphColor");
+            return v != null ? v : super.javaSubgraphColor();
         }
 
         @Override
         public String jdkSubgraphColor() {
-            return attrs.get("jdkSubgraphColor");
+            String v = attrs.getProperty("jdkSubgraphColor");
+            return v != null ? v : super.jdkSubgraphColor();
         }
 
+        @Override
+        public String nodeMargin() {
+            String v = attrs.getProperty("node-margin");
+            return v != null ? v : super.nodeMargin();
+        }
+
+        @Override
+        public String requiresStyle() {
+            String v = attrs.getProperty("requiresStyle");
+            return v != null ? v : super.requiresStyle();
+        };
+
+        @Override
+        public String requiresTransitiveStyle() {
+            String v = attrs.getProperty("requiresTransitiveStyle");
+            return v != null ? v : super.requiresTransitiveStyle();
+        };
+
         @Override
         public int weightOf(String s, String t) {
             int w = weights.getOrDefault(s + ":" + t, 1);
@@ -221,14 +232,25 @@ public class GenGraphs {
             return 1;
         }
 
-        public void weight(String s, String t, int w) {
-            weights.put(s + ":" + t, w);
-        }
+        /*
+         * Create a map of <mn>:<dep> with a weight trying to line up
+         * the modules in the weights property in the specified order.
+         */
+        public static Map<String, Integer> initWeights(Properties props) {
+            String[] modules = props.getProperty("weights", "").split(",");
+            int len = modules.length;
+            if (len == 0) return Map.of();
 
-        static Map<String, String> toAttributes(Properties props) {
-            return DEFAULT_ATTRIBUTES.keySet().stream()
-                .collect(Collectors.toMap(Function.identity(),
-                    k -> props.getProperty(k, DEFAULT_ATTRIBUTES.get(k))));
+            Map<String, Integer> weights = new HashMap<>();
+            String mn = modules[0];
+            int w = 10000;
+            for (int i = 1; i < len; i++) {
+                String dep = modules[i];
+                weights.put(mn + ":" + dep, w);
+                mn = dep;
+            }
+            weights.put(mn + ":java.base", w);
+            return weights;
         }
     }
 

make/jdk/src/classes/build/tools/jigsaw/javadoc-graphs.properties

@@ -1,2 +1,35 @@
+# Configuration file for build.tools.jigsaw.GenGraphs
+
+nodesep=.5
+node-margin=.2,.2
+ranksep=0.6
+fontsize=12
+fontcolor=#000000
+fontname=DejaVuSans
+arrowsize=1
+arrowwidth=2
+
+# requires edge: gray
 arrowcolor=#999999
-requiresMandatedColor=#999999
+
+# requires mandated java.base edge: light gray
+requiresMandatedColor=#dddddd
+
+requiresTransitiveStyle=
+requiresStyle=dashed
+
+# java.* modules: orange
+javaSubgraphColor=#e76f00
+
+# jdk.* modules: blue
+jdkSubgraphColor=#437291
+
+# configure the group of modules in the same rank
+ranks.1=java.logging,java.scripting,java.xml
+ranks.2=java.sql
+ranks.4=java.compiler,java.instrument
+ranks.5=java.desktop,java.management
+
+# configure the edges A -> B -> C .... with the same weight
+# that should get these modules lined in a straight line
+weights=java.se,java.sql.rowset,java.sql,java.xml

make/modules/java.base/Copy.gmk

@@ -95,16 +95,10 @@ ifeq ($(call And, $(call isTargetOs, windows) $(call isTargetCpu, x86)), true)
 endif
 DEFAULT_CFG_VARIANT ?= server
 
-# Any variant other than server, client or minimal is represented as server in
-# the cfg file.
-VALID_CFG_VARIANTS := server client minimal
-CFG_VARIANTS := $(filter $(VALID_CFG_VARIANTS), $(JVM_VARIANTS)) \
-    $(if $(filter-out $(VALID_CFG_VARIANTS), $(JVM_VARIANTS)), server)
-
 # Change the order to put the default variant first if present.
 ORDERED_CFG_VARIANTS := \
-    $(if $(filter $(DEFAULT_CFG_VARIANT), $(CFG_VARIANTS)), $(DEFAULT_CFG_VARIANT)) \
-    $(filter-out $(DEFAULT_CFG_VARIANT), $(CFG_VARIANTS))
+    $(if $(filter $(DEFAULT_CFG_VARIANT), $(JVM_VARIANTS)), $(DEFAULT_CFG_VARIANT)) \
+    $(filter-out $(DEFAULT_CFG_VARIANT), $(JVM_VARIANTS))
 
 JVMCFG := $(LIB_DST_DIR)/jvm.cfg
 

make/modules/java.base/Lib.gmk

@@ -156,11 +156,8 @@ ifeq ($(call isTargetOsType, unix), true)
         TARGETS += $(LIB_OUTPUTDIR)/$1/$(call SHARED_LIBRARY,jsig)
       endef
 
-      # The subdir is the same as the variant for client and minimal, for all
-      # others it's server.
-      VARIANT_SUBDIRS := $(filter client minimal, $(JVM_VARIANTS)) \
-          $(if $(filter-out client minimal, $(JVM_VARIANTS)), server)
-      $(foreach v, $(VARIANT_SUBDIRS), $(eval $(call CreateSymlinks,$v)))
+      # The subdir is the same as the variant
+      $(foreach v, $(JVM_VARIANTS), $(eval $(call CreateSymlinks,$v)))
     endif
     ############################################################################
 

make/test/BuildTestLib.gmk

@@ -36,9 +36,10 @@ TEST_LIB_SUPPORT := $(SUPPORT_OUTPUTDIR)/test/lib
 
 $(eval $(call SetupJavaCompilation, BUILD_WB_JAR, \
     TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \
-    SRC := $(TEST_LIB_SOURCE_DIR)/sun, \
+    SRC := $(TEST_LIB_SOURCE_DIR)/sun $(TEST_LIB_SOURCE_DIR)/jdk/test/whitebox/parser, \
     BIN := $(TEST_LIB_SUPPORT)/wb_classes, \
     JAR := $(TEST_LIB_SUPPORT)/wb.jar, \
+    DISABLED_WARNINGS := deprecation removal, \
 ))
 
 TARGETS += $(BUILD_WB_JAR)
@@ -50,7 +51,7 @@ $(eval $(call SetupJavaCompilation, BUILD_TEST_LIB_JAR, \
     BIN := $(TEST_LIB_SUPPORT)/test-lib_classes, \
     HEADERS := $(TEST_LIB_SUPPORT)/test-lib_headers, \
     JAR := $(TEST_LIB_SUPPORT)/test-lib.jar, \
-    DISABLED_WARNINGS := try deprecation rawtypes unchecked serial cast, \
+    DISABLED_WARNINGS := try deprecation rawtypes unchecked serial cast removal, \
 ))
 
 TARGETS += $(BUILD_TEST_LIB_JAR)

src/demo/share/README

@@ -4,12 +4,3 @@ deliberately simplified. Additional steps required for a
 production-quality application, such as security checks, input
 validation, and proper error handling, might not be present in the
 sample code.
-
-In some cases, the default security settings may block an execution
-of demo applets in a browser. To adjust the security settings, please
-refer to the following resource:
-
-http://java.com/en/download/help/java_blocked.xml
-
-Some demo applets need to be accessed through the HTTP or HTTPS
-protocols to enable access to the required resources.

src/demo/share/jfc/Font2DTest/Font2DTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 1999, 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 1999, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -72,6 +72,8 @@ import java.util.BitSet;
 import javax.swing.*;
 import javax.swing.event.*;
 
+import static java.nio.charset.StandardCharsets.UTF_16;
+
 /**
  * Font2DTest.java
  *
@@ -589,7 +591,7 @@ public final class Font2DTest extends JPanel
             if (numBytes >= 2 &&
                 (( byteData[0] == (byte) 0xFF && byteData[1] == (byte) 0xFE ) ||
                  ( byteData[0] == (byte) 0xFE && byteData[1] == (byte) 0xFF )))
-              fileText = new String( byteData, "UTF-16" );
+              fileText = new String(byteData, UTF_16);
             /// Otherwise, use system default encoding
             else
               fileText = new String( byteData );
@@ -647,7 +649,7 @@ public final class Font2DTest extends JPanel
                 showFontInfoCBMI.getState() + "\n" +
                 rm.getSelectedItem() + "\n" +
                 range[0] + "\n" + range[1] + "\n" + curOptions + tFileName);
-            byte[] toBeWritten = completeOptions.getBytes( "UTF-16" );
+            byte[] toBeWritten = completeOptions.getBytes(UTF_16);
             bos.write( toBeWritten, 0, toBeWritten.length );
             bos.close();
         }
@@ -712,7 +714,7 @@ public final class Font2DTest extends JPanel
                 (byteData[0] != (byte) 0xFE || byteData[1] != (byte) 0xFF) )
               throw new Exception( "Not a Font2DTest options file" );
 
-            String options = new String( byteData, "UTF-16" );
+            String options = new String(byteData, UTF_16);
             StringTokenizer perLine = new StringTokenizer( options, "\n" );
             String title = perLine.nextToken();
             if ( !title.equals( "Font2DTest Option File" ))

src/demo/share/jfc/Font2DTest/FontPanel.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2000, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -80,6 +80,7 @@ import javax.imageio.*;
 import javax.swing.*;
 
 import static java.awt.RenderingHints.*;
+import static java.nio.charset.StandardCharsets.ISO_8859_1;
 
 /**
  * FontPanel.java
@@ -643,7 +644,7 @@ public final class FontPanel extends JPanel implements AdjustmentListener {
                 break;
               case DRAW_BYTES:
                 try {
-                    byte[] lineBytes = line.getBytes( "ISO-8859-1" );
+                    byte[] lineBytes = line.getBytes(ISO_8859_1);
                     g2.drawBytes( lineBytes, 0, lineBytes.length, 0, 0 );
                 }
                 catch ( Exception e ) {

src/demo/share/jfc/J2Ddemo/java2d/J2Ddemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -109,7 +109,6 @@ public class J2Ddemo extends JPanel implements ItemListener, ActionListener, Dem
         { "Paths", "Append", "CurveQuadTo", "FillStroke", "WindingRule" },
         { "Transforms", "Rotate", "SelectTx", "TransformAnim" }
     };
-    private final boolean demoIsInApplet;
     private JCheckBoxMenuItem controlsCB;
     private JMenuItem runMI, cloneMI, fileMI, backgMI;
 //    private JMenuItem ccthreadMI, verboseMI;
@@ -122,8 +121,7 @@ public class J2Ddemo extends JPanel implements ItemListener, ActionListener, Dem
     /**
      * Construct the J2D Demo.
      */
-    public J2Ddemo(boolean demoIsInApplet, DemoProgress progress, RunWindowSettings runWndSetts) {
-        this.demoIsInApplet = demoIsInApplet;
+    public J2Ddemo(DemoProgress progress, RunWindowSettings runWndSetts) {
         this.runWndSetts = runWndSetts;
 
         setLayout(new BorderLayout());
@@ -171,11 +169,9 @@ public class J2Ddemo extends JPanel implements ItemListener, ActionListener, Dem
         JPopupMenu.setDefaultLightWeightPopupEnabled(false);
         JMenuBar menuBar = new JMenuBar();
 
-        if (!demoIsInApplet) {
-            JMenu file = menuBar.add(new JMenu("File"));
-            fileMI = file.add(new JMenuItem("Exit"));
-            fileMI.addActionListener(this);
-        }
+        JMenu file = menuBar.add(new JMenu("File"));
+        fileMI = file.add(new JMenuItem("Exit"));
+        fileMI.addActionListener(this);
 
         JMenu options = menuBar.add(new JMenu("Options"));
 
@@ -239,11 +235,7 @@ public class J2Ddemo extends JPanel implements ItemListener, ActionListener, Dem
         rf.addWindowListener(l);
         rf.getContentPane().add("Center", runwindow);
         rf.pack();
-        if (!demoIsInApplet) {
-            rf.setSize(new Dimension(200, 125));
-        } else {
-            rf.setSize(new Dimension(200, 150));
-        }
+        rf.setSize(new Dimension(200, 125));
         rf.setVisible(true);
     }
 
@@ -611,7 +603,7 @@ public class J2Ddemo extends JPanel implements ItemListener, ActionListener, Dem
 
         frame.setVisible(true);
 
-        J2Ddemo demo = new J2Ddemo(false, demoProgress, runWndSetts);
+        J2Ddemo demo = new J2Ddemo(demoProgress, runWndSetts);
         demoOneInstArr[0] = demo;
 
         frame.getContentPane().removeAll();

src/demo/share/jfc/SwingSet2/ButtonDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,30 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractButton;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.ButtonGroup;
+import javax.swing.JButton;
+import javax.swing.JCheckBox;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JRadioButton;
+import javax.swing.JTabbedPane;
+import javax.swing.JToggleButton;
+import javax.swing.SingleSelectionModel;
+import javax.swing.border.CompoundBorder;
+import javax.swing.border.EmptyBorder;
+import javax.swing.border.TitledBorder;
+import javax.swing.event.ChangeEvent;
+import javax.swing.event.ChangeListener;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.Component;
+import java.awt.Dimension;
+import java.awt.Insets;
+import java.awt.event.ItemEvent;
+import java.awt.event.ItemListener;
+import java.util.Vector;
 
 /**
  * JButton, JRadioButton, JToggleButton, JCheckBox Demos

src/demo/share/jfc/SwingSet2/ColorChooserDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,20 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.Icon;
+import javax.swing.JButton;
+import javax.swing.JColorChooser;
+import javax.swing.JDialog;
+import javax.swing.JPanel;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.Color;
+import java.awt.Component;
+import java.awt.Dimension;
+import java.awt.Graphics;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
 
 /**
  * JColorChooserDemo

src/demo/share/jfc/SwingSet2/ComboBoxDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,23 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.accessibility.AccessibleRelation;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.Icon;
+import javax.swing.ImageIcon;
+import javax.swing.JComboBox;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.border.BevelBorder;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Component;
+import java.awt.Dimension;
+import java.awt.Graphics;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
+import java.util.Hashtable;
 
 /**
  * JComboBox Demo

src/demo/share/jfc/SwingSet2/DemoModule.java

@@ -1,6 +1,5 @@
 /*
- *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,21 +29,25 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
+import javax.swing.BoxLayout;
+import javax.swing.Icon;
+import javax.swing.ImageIcon;
+import javax.swing.JFrame;
+import javax.swing.JPanel;
+import javax.swing.UIManager;
+import javax.swing.border.Border;
+import javax.swing.border.CompoundBorder;
+import javax.swing.border.EmptyBorder;
+import javax.swing.border.SoftBevelBorder;
 
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Dimension;
+import java.io.BufferedReader;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
 
 /**
  * A generic SwingSet2 demo module
@@ -155,7 +158,7 @@ public class DemoModule extends JFrame {
             try {
                 url = getClass().getResource(filename);
                 is = url.openStream();
-                isr = new InputStreamReader(is, "UTF-8");
+                isr = new InputStreamReader(is, UTF_8);
                 BufferedReader reader = new BufferedReader(isr);
 
                 // Read one line at a time, htmlize using super-spiffy

src/demo/share/jfc/SwingSet2/FileChooserDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,37 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractAction;
+import javax.swing.Action;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.Icon;
+import javax.swing.ImageIcon;
+import javax.swing.JButton;
+import javax.swing.JComponent;
+import javax.swing.JDialog;
+import javax.swing.JFileChooser;
+import javax.swing.JLabel;
+import javax.swing.JOptionPane;
+import javax.swing.JPanel;
+import javax.swing.JScrollPane;
+import javax.swing.SwingUtilities;
+import javax.swing.WindowConstants;
+import javax.swing.border.BevelBorder;
+import javax.swing.filechooser.FileNameExtensionFilter;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Color;
+import java.awt.Component;
+import java.awt.Dimension;
+import java.awt.Frame;
+import java.awt.Graphics;
+import java.awt.Image;
+import java.awt.Insets;
+import java.awt.event.ActionEvent;
+import java.beans.PropertyChangeEvent;
+import java.beans.PropertyChangeListener;
+import java.io.File;
 
 /**
  * JFileChooserDemo

src/demo/share/jfc/SwingSet2/HtmlDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,23 +30,19 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.JEditorPane;
+import javax.swing.JScrollPane;
+import javax.swing.JViewport;
+import javax.swing.event.HyperlinkEvent.EventType;
+import javax.swing.event.HyperlinkEvent;
+import javax.swing.event.HyperlinkListener;
+import javax.swing.text.html.HTMLDocument;
+import javax.swing.text.html.HTMLFrameHyperlinkEvent;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.text.html.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
 
 /**
  * Html Demo

src/demo/share/jfc/SwingSet2/InternalFrameDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,26 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractAction;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.Icon;
+import javax.swing.ImageIcon;
+import javax.swing.JButton;
+import javax.swing.JCheckBox;
+import javax.swing.JDesktopPane;
+import javax.swing.JInternalFrame;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JScrollPane;
+import javax.swing.JTextField;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Color;
+import java.awt.Dimension;
+import java.awt.GridLayout;
+import java.awt.Insets;
+import java.awt.event.ActionEvent;
 
 /**
  * Internal Frames Demo

src/demo/share/jfc/SwingSet2/ListDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,29 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractAction;
+import javax.swing.AbstractListModel;
+import javax.swing.Action;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.DefaultListCellRenderer;
+import javax.swing.ImageIcon;
+import javax.swing.JCheckBox;
+import javax.swing.JComponent;
+import javax.swing.JLabel;
+import javax.swing.JList;
+import javax.swing.JPanel;
+import javax.swing.JScrollPane;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Component;
+import java.awt.Insets;
+import java.awt.Rectangle;
+import java.awt.event.ActionEvent;
+import java.awt.event.FocusAdapter;
+import java.awt.event.FocusEvent;
+import java.awt.event.FocusListener;
+import java.util.Vector;
 
 /**
  * List Demo. This demo shows that it is not

src/demo/share/jfc/SwingSet2/OptionPaneDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,19 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractAction;
+import javax.swing.Action;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.JButton;
+import javax.swing.JComboBox;
+import javax.swing.JOptionPane;
+import javax.swing.JPanel;
+import javax.swing.JTextField;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.Dimension;
+import java.awt.event.ActionEvent;
+import java.net.URL;
 
 /**
  * JOptionPaneDemo

src/demo/share/jfc/SwingSet2/ProgressBarDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,19 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.AbstractAction;
+import javax.swing.Action;
+import javax.swing.JButton;
+import javax.swing.JPanel;
+import javax.swing.JProgressBar;
+import javax.swing.JScrollPane;
+import javax.swing.JTextArea;
+import javax.swing.border.BevelBorder;
+import javax.swing.border.SoftBevelBorder;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Dimension;
+import java.awt.event.ActionEvent;
 
 /**
  * JProgressBar Demo

src/demo/share/jfc/SwingSet2/README.txt

@@ -10,12 +10,6 @@ TO RUN SWINGSET2 AS AN APPLICATION
   java -jar SwingSet2.jar
 
 
-=============================
-TO RUN SWINGSET2 AS AN APPLET
-=============================
-
-  appletviewer SwingSet2.html
-
 =========================
 TO MODIFY/BUILD SWINGSET2
 =========================
@@ -40,8 +34,8 @@ documentation) or you can specify the splash screen image on the command line:
 java -splash:resources/images/splash.png SwingSet2
 
 
-Note: These instructions assume that this installation's versions of the java,
-appletviewer, and javac commands are in your path.  If they aren't, then you should
+Note: These instructions assume that this installation's versions of the java
+and javac commands are in your path.  If they aren't, then you should
 either specify the complete path to the commands or update your PATH environment
 variable as described in the installation instructions for the
 Java(TM) SE Development Kit.

src/demo/share/jfc/SwingSet2/ScrollPaneDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,14 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.Icon;
+import javax.swing.ImageIcon;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JScrollBar;
+import javax.swing.JScrollPane;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
 
 /**
  * Scroll Pane Demo

src/demo/share/jfc/SwingSet2/SliderDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,21 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.BoundedRangeModel;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.DefaultBoundedRangeModel;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JSlider;
+import javax.swing.border.BevelBorder;
+import javax.swing.border.TitledBorder;
+import javax.swing.event.ChangeEvent;
+import javax.swing.event.ChangeListener;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.GridLayout;
+import java.util.Dictionary;
 
 /**
  * JSlider Demo

src/demo/share/jfc/SwingSet2/SplitPaneDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,23 +30,26 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.ButtonGroup;
+import javax.swing.JCheckBox;
+import javax.swing.JLabel;
+import javax.swing.JOptionPane;
+import javax.swing.JPanel;
+import javax.swing.JRadioButton;
+import javax.swing.JSplitPane;
+import javax.swing.JTextField;
+import javax.swing.event.ChangeEvent;
+import javax.swing.event.ChangeListener;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.table.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Color;
+import java.awt.Dimension;
+import java.awt.FlowLayout;
+import java.awt.GridLayout;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
 
 /**
  * Split Pane demo

src/demo/share/jfc/SwingSet2/SwingSet2.html

@@ -1,14 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
-<html>
-  <head>
-    <title>SwingSet demo</title>
-  </head>
-
-  <body>
-      <h1>SwingSet demo</h1>
-      <applet code=SwingSet2Applet              
-              archive="SwingSet2.jar"
-      	      width=695 height=525>
-      </applet>
-  </body>
-</html>

src/demo/share/jfc/SwingSet2/TabbedPaneDemo.java

@@ -1,6 +1,5 @@
 /*
- *
- * Copyright (c) 2007, 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +29,23 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.ButtonGroup;
+import javax.swing.ImageIcon;
+import javax.swing.JComponent;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JRadioButton;
+import javax.swing.JTabbedPane;
+import javax.swing.SingleSelectionModel;
+import javax.swing.event.ChangeEvent;
+import javax.swing.event.ChangeListener;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.BorderLayout;
+import java.awt.Color;
+import java.awt.Graphics;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
+import java.util.Random;
 
 /**
  * JTabbedPane Demo

src/demo/share/jfc/SwingSet2/TableDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,24 +30,51 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.accessibility.Accessible;
+import javax.accessibility.AccessibleContext;
+import javax.accessibility.AccessibleRelation;
+import javax.accessibility.AccessibleRelationSet;
+import javax.swing.AbstractAction;
+import javax.swing.BorderFactory;
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.DefaultCellEditor;
+import javax.swing.ImageIcon;
+import javax.swing.JButton;
+import javax.swing.JCheckBox;
+import javax.swing.JComboBox;
+import javax.swing.JComponent;
+import javax.swing.JLabel;
+import javax.swing.JOptionPane;
+import javax.swing.JPanel;
+import javax.swing.JScrollPane;
+import javax.swing.JSlider;
+import javax.swing.JTable;
+import javax.swing.JTextField;
+import javax.swing.KeyStroke;
+import javax.swing.border.TitledBorder;
+import javax.swing.event.ChangeEvent;
+import javax.swing.event.ChangeListener;
+import javax.swing.table.AbstractTableModel;
+import javax.swing.table.DefaultTableCellRenderer;
+import javax.swing.table.TableColumn;
+import javax.swing.table.TableModel;
+import javax.swing.table.TableRowSorter;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.table.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
+import java.awt.BorderLayout;
+import java.awt.Color;
+import java.awt.Component;
+import java.awt.Container;
+import java.awt.Dimension;
+import java.awt.GridLayout;
+import java.awt.Insets;
+import java.awt.LayoutManager;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
+import java.awt.event.ItemEvent;
+import java.awt.event.ItemListener;
 import java.awt.print.PrinterException;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.util.Vector;
 
 import java.text.MessageFormat;
 

src/demo/share/jfc/SwingSet2/ToolTipDemo.java

@@ -1,6 +1,6 @@
 /*
  *
- * Copyright (c) 2007, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,22 +30,15 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.Box;
+import javax.swing.BoxLayout;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.text.*;
-import javax.swing.border.*;
-import javax.swing.colorchooser.*;
-import javax.swing.filechooser.*;
-import javax.accessibility.*;
-
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import java.awt.Color;
+import java.awt.Dimension;
+import java.awt.Point;
+import java.awt.Polygon;
 
 /**
  * ToolTip Demo

src/demo/share/jfc/SwingSet2/TreeDemo.java

@@ -1,6 +1,5 @@
 /*
- *
- * Copyright (c) 2007, 2019, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2007, 2021, Oracle and/or its affiliates. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -30,19 +29,19 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+import javax.swing.JScrollPane;
+import javax.swing.JTree;
+import javax.swing.tree.DefaultMutableTreeNode;
 
-import javax.swing.*;
-import javax.swing.event.*;
-import javax.swing.tree.*;
-import javax.accessibility.*;
+import java.awt.BorderLayout;
+import java.awt.Insets;
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
 
-import java.awt.*;
-import java.awt.event.*;
-import java.beans.*;
-import java.util.*;
-import java.io.*;
-import java.applet.*;
-import java.net.*;
+import static java.nio.charset.StandardCharsets.UTF_8;
 
 /**
  * JTree Demo
@@ -84,7 +83,7 @@ public class TreeDemo extends DemoModule {
         try {
             // convert url to buffered string
             InputStream is = url.openStream();
-            InputStreamReader isr = new InputStreamReader(is, "UTF-8");
+            InputStreamReader isr = new InputStreamReader(is, UTF_8);
             BufferedReader reader = new BufferedReader(isr);
 
             // read one line at a time, put into tree

src/hotspot/cpu/aarch64/aarch64.ad

@@ -1902,7 +1902,7 @@ void MachPrologNode::emit(CodeBuffer &cbuf, PhaseRegAlloc *ra_) const {
     __ bind(L_skip_barrier);
   }
 
-  if (C->max_vector_size() >= 16) {
+  if (C->max_vector_size() > 0) {
     __ reinitialize_ptrue();
   }
 
@@ -2388,7 +2388,7 @@ const bool Matcher::match_rule_supported(int opcode) {
 // Identify extra cases that we might want to provide match rules for vector nodes and
 // other intrinsics guarded with vector length (vlen) and element type (bt).
 const bool Matcher::match_rule_supported_vector(int opcode, int vlen, BasicType bt) {
-  if (!match_rule_supported(opcode) || !vector_size_supported(bt, vlen)) {
+  if (!match_rule_supported(opcode)) {
     return false;
   }
   int bit_size = vlen * type2aelembytes(bt) * 8;
@@ -2396,7 +2396,7 @@ const bool Matcher::match_rule_supported_vector(int opcode, int vlen, BasicType
     return false;
   }
   if (UseSVE > 0) {
-    return op_sve_supported(opcode);
+    return op_sve_supported(opcode, vlen, bt);
   } else { // NEON
     // Special cases
     switch (opcode) {
@@ -2438,11 +2438,14 @@ const bool Matcher::match_rule_supported_vector(int opcode, int vlen, BasicType
         return false;
       }
       break;
+    case Op_LoadVectorGather:
+    case Op_StoreVectorScatter:
+      return false;
     default:
       break;
     }
   }
-  return true; // Per default match rules are supported.
+  return vector_size_supported(bt, vlen);
 }
 
 const RegMask* Matcher::predicate_reg_mask(void) {
@@ -2488,24 +2491,20 @@ const int Matcher::vector_width_in_bytes(BasicType bt) {
 const int Matcher::max_vector_size(const BasicType bt) {
   return vector_width_in_bytes(bt)/type2aelembytes(bt);
 }
+
 const int Matcher::min_vector_size(const BasicType bt) {
   int max_size = max_vector_size(bt);
-  if ((UseSVE > 0) && (MaxVectorSize >= 16)) {
-    // Currently vector length less than SVE vector register size is not supported.
-    return max_size;
-  } else { // NEON
-    // Limit the vector size to 8 bytes
-    int size = 8 / type2aelembytes(bt);
-    if (bt == T_BYTE) {
-      // To support vector api shuffle/rearrange.
-      size = 4;
-    } else if (bt == T_BOOLEAN) {
-      // To support vector api load/store mask.
-      size = 2;
-    }
-    if (size < 2) size = 2;
-    return MIN2(size,max_size);
+  // Limit the min vector size to 8 bytes.
+  int size = 8 / type2aelembytes(bt);
+  if (bt == T_BYTE) {
+    // To support vector api shuffle/rearrange.
+    size = 4;
+  } else if (bt == T_BOOLEAN) {
+    // To support vector api load/store mask.
+    size = 2;
   }
+  if (size < 2) size = 2;
+  return MIN2(size, max_size);
 }
 
 // Actual max scalable vector register length.
@@ -2515,7 +2514,7 @@ const int Matcher::scalable_vector_reg_size(const BasicType bt) {
 
 // Vector ideal reg.
 const uint Matcher::vector_ideal_reg(int len) {
-  if (UseSVE > 0 && 16 <= len && len <= 256) {
+  if (UseSVE > 0 && 2 <= len && len <= 256) {
     return Op_VecA;
   }
   switch(len) {
@@ -2642,6 +2641,13 @@ const RegMask Matcher::method_handle_invoke_SP_save_mask() {
 bool size_fits_all_mem_uses(AddPNode* addp, int shift) {
   for (DUIterator_Fast imax, i = addp->fast_outs(imax); i < imax; i++) {
     Node* u = addp->fast_out(i);
+    if (u->is_LoadStore()) {
+      // On AArch64, LoadStoreNodes (i.e. compare and swap
+      // instructions) only take register indirect as an operand, so
+      // any attempt to use an AddPNode as an input to a LoadStoreNode
+      // must fail.
+      return false;
+    }
     if (u->is_Mem()) {
       int opsize = u->as_Mem()->memory_size();
       assert(opsize > 0, "unexpected memory operand size");
@@ -3713,7 +3719,7 @@ encode %{
     }
 
     // Only non uncommon_trap calls need to reinitialize ptrue.
-    if (Compile::current()->max_vector_size() >= 16 && uncommon_trap_request() == 0) {
+    if (Compile::current()->max_vector_size() > 0 && uncommon_trap_request() == 0) {
       __ reinitialize_ptrue();
     }
   %}
@@ -3725,7 +3731,7 @@ encode %{
     if (call == NULL) {
       ciEnv::current()->record_failure("CodeCache is full");
       return;
-    } else if (Compile::current()->max_vector_size() >= 16) {
+    } else if (Compile::current()->max_vector_size() > 0) {
       __ reinitialize_ptrue();
     }
   %}
@@ -3763,7 +3769,7 @@ encode %{
       __ bind(retaddr);
       __ add(sp, sp, 2 * wordSize);
     }
-    if (Compile::current()->max_vector_size() >= 16) {
+    if (Compile::current()->max_vector_size() > 0) {
       __ reinitialize_ptrue();
     }
   %}
@@ -3776,7 +3782,7 @@ encode %{
   enc_class aarch64_enc_ret() %{
     C2_MacroAssembler _masm(&cbuf);
 #ifdef ASSERT
-    if (Compile::current()->max_vector_size() >= 16) {
+    if (Compile::current()->max_vector_size() > 0) {
       __ verify_ptrue();
     }
 #endif
@@ -4149,6 +4155,16 @@ operand immIExt()
   interface(CONST_INTER);
 %}
 
+operand immI_gt_1()
+%{
+  predicate(n->get_int() > 1);
+  match(ConI);
+
+  op_cost(0);
+  format %{ %}
+  interface(CONST_INTER);
+%}
+
 operand immI_le_4()
 %{
   predicate(n->get_int() <= 4);
@@ -10877,7 +10893,6 @@ instruct msubI(iRegINoSp dst, iRegIorL2I src1, iRegIorL2I src2, iRegIorL2I src3)
 
 instruct mnegI(iRegINoSp dst, iRegIorL2I src1, iRegIorL2I src2, immI0 zero) %{
   match(Set dst (MulI (SubI zero src1) src2));
-  match(Set dst (MulI src1 (SubI zero src2)));
 
   ins_cost(INSN_COST * 3);
   format %{ "mneg  $dst, $src1, $src2" %}
@@ -10929,7 +10944,6 @@ instruct msubL(iRegLNoSp dst, iRegL src1, iRegL src2, iRegL src3) %{
 
 instruct mnegL(iRegLNoSp dst, iRegL src1, iRegL src2, immL0 zero) %{
   match(Set dst (MulL (SubL zero src1) src2));
-  match(Set dst (MulL src1 (SubL zero src2)));
 
   ins_cost(INSN_COST * 5);
   format %{ "mneg  $dst, $src1, $src2" %}
@@ -10979,7 +10993,6 @@ instruct smsubL(iRegLNoSp dst, iRegIorL2I src1, iRegIorL2I src2, iRegLNoSp src3)
 
 instruct smnegL(iRegLNoSp dst, iRegIorL2I src1, iRegIorL2I src2, immL0 zero) %{
   match(Set dst (MulL (SubL zero (ConvI2L src1)) (ConvI2L src2)));
-  match(Set dst (MulL (ConvI2L src1) (SubL zero (ConvI2L src2))));
 
   ins_cost(INSN_COST * 3);
   format %{ "smnegl  $dst, $src1, $src2" %}
@@ -16848,6 +16861,7 @@ instruct encode_iso_array(iRegP_R2 src, iRegP_R1 dst, iRegI_R3 len,
                           vRegD_V2 Vtmp3, vRegD_V3 Vtmp4,
                           iRegI_R0 result, rFlagsReg cr)
 %{
+  predicate(!((EncodeISOArrayNode*)n)->is_ascii());
   match(Set result (EncodeISOArray src (Binary dst len)));
   effect(USE_KILL src, USE_KILL dst, USE_KILL len,
          KILL Vtmp1, KILL Vtmp2, KILL Vtmp3, KILL Vtmp4, KILL cr);

src/hotspot/cpu/aarch64/aarch64_neon.ad

@@ -33,7 +33,7 @@
 // Load Vector (16 bits)
 instruct loadV2(vecD dst, vmem2 mem)
 %{
-  predicate(n->as_LoadVector()->memory_size() == 2);
+  predicate(UseSVE == 0 && n->as_LoadVector()->memory_size() == 2);
   match(Set dst (LoadVector mem));
   ins_cost(4 * INSN_COST);
   format %{ "ldrh   $dst,$mem\t# vector (16 bits)" %}
@@ -44,7 +44,7 @@ instruct loadV2(vecD dst, vmem2 mem)
 // Load Vector (32 bits)
 instruct loadV4(vecD dst, vmem4 mem)
 %{
-  predicate(n->as_LoadVector()->memory_size() == 4);
+  predicate(UseSVE == 0 && n->as_LoadVector()->memory_size() == 4);
   match(Set dst (LoadVector mem));
   ins_cost(4 * INSN_COST);
   format %{ "ldrs   $dst,$mem\t# vector (32 bits)" %}
@@ -55,7 +55,7 @@ instruct loadV4(vecD dst, vmem4 mem)
 // Load Vector (64 bits)
 instruct loadV8(vecD dst, vmem8 mem)
 %{
-  predicate(n->as_LoadVector()->memory_size() == 8);
+  predicate(UseSVE == 0 && n->as_LoadVector()->memory_size() == 8);
   match(Set dst (LoadVector mem));
   ins_cost(4 * INSN_COST);
   format %{ "ldrd   $dst,$mem\t# vector (64 bits)" %}
@@ -2473,9 +2473,10 @@ instruct vmaskcastX(vecX dst)
 
 instruct loadcon8B(vecD dst, immI0 src)
 %{
-  predicate((n->as_Vector()->length() == 2 || n->as_Vector()->length() == 4 ||
-             n->as_Vector()->length() == 8) &&
-             n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);
+  predicate(UseSVE == 0 &&
+           (n->as_Vector()->length() == 2 || n->as_Vector()->length() == 4 ||
+            n->as_Vector()->length() == 8) &&
+            n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);
   match(Set dst (VectorLoadConst src));
   ins_cost(INSN_COST);
   format %{ "ldr $dst, CONSTANT_MEMORY\t# load iota indices" %}
@@ -2488,7 +2489,7 @@ instruct loadcon8B(vecD dst, immI0 src)
 
 instruct loadcon16B(vecX dst, immI0 src)
 %{
-  predicate(n->as_Vector()->length() == 16 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);
+  predicate(UseSVE == 0 && n->as_Vector()->length() == 16 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);
   match(Set dst (VectorLoadConst src));
   ins_cost(INSN_COST);
   format %{ "ldr $dst, CONSTANT_MEMORY\t# load iota indices" %}
@@ -2945,8 +2946,8 @@ instruct vabd2D(vecX dst, vecX src1, vecX src2)
 
 instruct replicate8B(vecD dst, iRegIorL2I src)
 %{
-  predicate(n->as_Vector()->length() == 4 ||
-            n->as_Vector()->length() == 8);
+  predicate(UseSVE == 0 && (n->as_Vector()->length() == 8 ||
+                            n->as_Vector()->length() == 4));
   match(Set dst (ReplicateB src));
   ins_cost(INSN_COST);
   format %{ "dup  $dst, $src\t# vector (8B)" %}
@@ -2970,8 +2971,8 @@ instruct replicate16B(vecX dst, iRegIorL2I src)
 
 instruct replicate8B_imm(vecD dst, immI con)
 %{
-  predicate(n->as_Vector()->length() == 4 ||
-            n->as_Vector()->length() == 8);
+  predicate(UseSVE == 0 && (n->as_Vector()->length() == 8 ||
+                            n->as_Vector()->length() == 4));
   match(Set dst (ReplicateB con));
   ins_cost(INSN_COST);
   format %{ "movi  $dst, $con\t# vector (8B)" %}
@@ -2995,8 +2996,8 @@ instruct replicate16B_imm(vecX dst, immI con)
 
 instruct replicate4S(vecD dst, iRegIorL2I src)
 %{
-  predicate(n->as_Vector()->length() == 2 ||
-            n->as_Vector()->length() == 4);
+  predicate(UseSVE == 0 && (n->as_Vector()->length() == 4 ||
+                            n->as_Vector()->length() == 2));
   match(Set dst (ReplicateS src));
   ins_cost(INSN_COST);
   format %{ "dup  $dst, $src\t# vector (4S)" %}
@@ -3020,8 +3021,8 @@ instruct replicate8S(vecX dst, iRegIorL2I src)
 
 instruct replicate4S_imm(vecD dst, immI con)
 %{
-  predicate(n->as_Vector()->length() == 2 ||
-            n->as_Vector()->length() == 4);
+  predicate(UseSVE == 0 && (n->as_Vector()->length() == 4 ||
+                            n->as_Vector()->length() == 2));
   match(Set dst (ReplicateS con));
   ins_cost(INSN_COST);
   format %{ "movi  $dst, $con\t# vector (4H)" %}
@@ -3045,7 +3046,7 @@ instruct replicate8S_imm(vecX dst, immI con)
 
 instruct replicate2I(vecD dst, iRegIorL2I src)
 %{
-  predicate(n->as_Vector()->length() == 2);
+  predicate(UseSVE == 0 && n->as_Vector()->length() == 2);
   match(Set dst (ReplicateI src));
   ins_cost(INSN_COST);
   format %{ "dup  $dst, $src\t# vector (2I)" %}
@@ -3069,7 +3070,7 @@ instruct replicate4I(vecX dst, iRegIorL2I src)
 
 instruct replicate2I_imm(vecD dst, immI con)
 %{
-  predicate(n->as_Vector()->length() == 2);
+  predicate(UseSVE == 0 && n->as_Vector()->length() == 2);
   match(Set dst (ReplicateI con));
   ins_cost(INSN_COST);
   format %{ "movi  $dst, $con\t# vector (2I)" %}
@@ -3119,7 +3120,7 @@ instruct replicate2L_zero(vecX dst, immI0 zero)
 
 instruct replicate2F(vecD dst, vRegF src)
 %{
-  predicate(n->as_Vector()->length() == 2);
+  predicate(UseSVE == 0 && n->as_Vector()->length() == 2);
   match(Set dst (ReplicateF src));
   ins_cost(INSN_COST);
   format %{ "dup  $dst, $src\t# vector (2F)" %}
@@ -4249,8 +4250,8 @@ instruct vxor16B(vecX dst, vecX src1, vecX src2)
 // ------------------------------ Shift ---------------------------------------
 
 instruct vshiftcnt8B(vecD dst, iRegIorL2I cnt) %{
-  predicate(n->as_Vector()->length_in_bytes() == 4 ||
-            n->as_Vector()->length_in_bytes() == 8);
+  predicate(UseSVE == 0 && (n->as_Vector()->length_in_bytes() == 4 ||
+            n->as_Vector()->length_in_bytes() == 8));
   match(Set dst (LShiftCntV cnt));
   match(Set dst (RShiftCntV cnt));
   format %{ "dup  $dst, $cnt\t# shift count vector (8B)" %}
@@ -4261,7 +4262,7 @@ instruct vshiftcnt8B(vecD dst, iRegIorL2I cnt) %{
 %}
 
 instruct vshiftcnt16B(vecX dst, iRegIorL2I cnt) %{
-  predicate(n->as_Vector()->length_in_bytes() == 16);
+  predicate(UseSVE == 0 && (n->as_Vector()->length_in_bytes() == 16));
   match(Set dst (LShiftCntV cnt));
   match(Set dst (RShiftCntV cnt));
   format %{ "dup  $dst, $cnt\t# shift count vector (16B)" %}

src/hotspot/cpu/aarch64/aarch64_neon_ad.m4

@@ -69,9 +69,9 @@ instruct $3V$4`'(vec$5 $7, vmem$4 mem)
   ins_pipe(v$3`_reg_mem'ifelse(eval($4 * 8), 128, 128, 64));
 %}')dnl
 dnl        $1    $2 $3     $4  $5 $6   $7   $8
-VLoadStore(ldrh, H, load,  2,  D, 16,  dst, )
-VLoadStore(ldrs, S, load,  4,  D, 32,  dst, )
-VLoadStore(ldrd, D, load,  8,  D, 64,  dst, )
+VLoadStore(ldrh, H, load,  2,  D, 16,  dst, UseSVE == 0 && )
+VLoadStore(ldrs, S, load,  4,  D, 32,  dst, UseSVE == 0 && )
+VLoadStore(ldrd, D, load,  8,  D, 64,  dst, UseSVE == 0 && )
 VLoadStore(ldrq, Q, load, 16,  X, 128, dst, UseSVE == 0 && )
 VLoadStore(strh, H, store, 2,  D, 16,  src, )
 VLoadStore(strs, S, store, 4,  D, 32,  src, )
@@ -1196,10 +1196,11 @@ dnl
 //-------------------------------- LOAD_IOTA_INDICES----------------------------------
 dnl
 define(`PREDICATE', `ifelse($1, 8,
-`predicate((n->as_Vector()->length() == 2 || n->as_Vector()->length() == 4 ||
-             n->as_Vector()->length() == 8) &&
-             n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);',
-`predicate(n->as_Vector()->length() == 16 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);')')dnl
+`predicate(UseSVE == 0 &&
+           (n->as_Vector()->length() == 2 || n->as_Vector()->length() == 4 ||
+            n->as_Vector()->length() == 8) &&
+            n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);',
+`predicate(UseSVE == 0 && n->as_Vector()->length() == 16 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE);')')dnl
 dnl
 define(`VECTOR_LOAD_CON', `
 instruct loadcon$1B`'(vec$2 dst, immI0 src)
@@ -1466,9 +1467,10 @@ dnl
 define(`VREPLICATE', `
 instruct replicate$3$4$5`'(vec$6 dst, $7 ifelse($7, immI0, zero, $7, immI, con, src))
 %{
-  predicate(ifelse($8, UseSVE == 0 && , $8,
-                   $8, , , $8`
-            ')n->as_Vector()->length() == $3);
+  predicate(UseSVE == 0 && ifelse($8, `',
+                                  n->as_Vector()->length() == $3,
+                                  (n->as_Vector()->length() == $3 ||`
+                            'n->as_Vector()->length() == $8)));
   match(Set dst (Replicate`'ifelse($7, immI0, I, $4) ifelse($7, immI0, zero, $7, immI, con, $7, zero, I, src)));
   ins_cost(INSN_COST);
   format %{ "$1  $dst, $ifelse($7, immI0, zero, $7, immI, con, src)`\t# vector ('ifelse($4$7, SimmI, $3H, $2, eor, 4I, $3$4)`)"' %}
@@ -1494,24 +1496,24 @@ instruct replicate$3$4$5`'(vec$6 dst, $7 ifelse($7, immI0, zero, $7, immI, con,
                   $7, iRegL, vdup_reg_reg,
                   $4, F, vdup_reg_freg, vdup_reg_dreg)`'ifelse($6, X, 128, 64));
 %}')dnl
-dnl        $1    $2    $3  $4 $5     $6 $7          $8                                $9
-VREPLICATE(dup,  dup,  8,  B, ,      D, iRegIorL2I, n->as_Vector()->length() == 4 ||, B)
-VREPLICATE(dup,  dup,  16, B, ,      X, iRegIorL2I, UseSVE == 0 && ,                  B)
-VREPLICATE(movi, mov,  8,  B, _imm,  D, immI,       n->as_Vector()->length() == 4 ||, B)
-VREPLICATE(movi, mov,  16, B, _imm,  X, immI,       UseSVE == 0 && ,                  B)
-VREPLICATE(dup,  dup,  4,  S, ,      D, iRegIorL2I, n->as_Vector()->length() == 2 ||, H)
-VREPLICATE(dup,  dup,  8,  S, ,      X, iRegIorL2I, UseSVE == 0 && ,                  H)
-VREPLICATE(movi, mov,  4,  S, _imm,  D, immI,       n->as_Vector()->length() == 2 ||, H)
-VREPLICATE(movi, mov,  8,  S,  _imm, X, immI,       UseSVE == 0 && ,                  H)
-VREPLICATE(dup,  dup,  2,  I, ,      D, iRegIorL2I, ,                                 S)
-VREPLICATE(dup,  dup,  4,  I, ,      X, iRegIorL2I, UseSVE == 0 && ,                  S)
-VREPLICATE(movi, mov,  2,  I, _imm,  D, immI,       ,                                 S)
-VREPLICATE(movi, mov,  4,  I,  _imm, X, immI,       UseSVE == 0 && ,                  S)
-VREPLICATE(dup,  dup,  2,  L, ,      X, iRegL,      UseSVE == 0 && ,                  D)
-VREPLICATE(movi, eor,  2,  L, _zero, X, immI0,      UseSVE == 0 && ,                  D)
-VREPLICATE(dup,  dup,  2,  F, ,      D, vRegF,      ,                                 S)
-VREPLICATE(dup,  dup,  4,  F, ,      X, vRegF,      UseSVE == 0 && ,                  S)
-VREPLICATE(dup,  dup,  2,  D, ,      X, vRegD,      UseSVE == 0 && ,                  D)
+dnl        $1    $2    $3  $4 $5     $6 $7          $8 $9
+VREPLICATE(dup,  dup,  8,  B, ,      D, iRegIorL2I, 4, B)
+VREPLICATE(dup,  dup,  16, B, ,      X, iRegIorL2I,  , B)
+VREPLICATE(movi, mov,  8,  B, _imm,  D, immI,       4, B)
+VREPLICATE(movi, mov,  16, B, _imm,  X, immI,        , B)
+VREPLICATE(dup,  dup,  4,  S, ,      D, iRegIorL2I, 2, H)
+VREPLICATE(dup,  dup,  8,  S, ,      X, iRegIorL2I,  , H)
+VREPLICATE(movi, mov,  4,  S, _imm,  D, immI,       2, H)
+VREPLICATE(movi, mov,  8,  S,  _imm, X, immI,        , H)
+VREPLICATE(dup,  dup,  2,  I, ,      D, iRegIorL2I, ,  S)
+VREPLICATE(dup,  dup,  4,  I, ,      X, iRegIorL2I, ,  S)
+VREPLICATE(movi, mov,  2,  I, _imm,  D, immI,       ,  S)
+VREPLICATE(movi, mov,  4,  I,  _imm, X, immI,       ,  S)
+VREPLICATE(dup,  dup,  2,  L, ,      X, iRegL,      ,  D)
+VREPLICATE(movi, eor,  2,  L, _zero, X, immI0,      ,  D)
+VREPLICATE(dup,  dup,  2,  F, ,      D, vRegF,      ,  S)
+VREPLICATE(dup,  dup,  4,  F, ,      X, vRegF,      ,  S)
+VREPLICATE(dup,  dup,  2,  D, ,      X, vRegD,      ,  D)
 dnl
 
 // ====================REDUCTION ARITHMETIC====================================
@@ -1884,8 +1886,8 @@ VLOGICAL(xor, eor,  xor, Xor, 16, B, X)
 dnl
 define(`VSHIFTCNT', `
 instruct vshiftcnt$3$4`'(vec$5 dst, iRegIorL2I cnt) %{
-  predicate(ifelse($3, 8, n->as_Vector()->length_in_bytes() == 4 ||`
-            ')n->as_Vector()->length_in_bytes() == $3);
+  predicate(UseSVE == 0 && (ifelse($3, 8, n->as_Vector()->length_in_bytes() == 4 ||`
+            ')n->as_Vector()->length_in_bytes() == $3));
   match(Set dst (LShiftCntV cnt));
   match(Set dst (RShiftCntV cnt));
   format %{ "$1  $dst, $cnt\t# shift count vector ($3$4)" %}

src/hotspot/cpu/aarch64/assembler_aarch64.cpp

@@ -54,10 +54,32 @@ Assembler::SIMD_Arrangement Assembler::_esize2arrangement_table[9][2] = {
   /*   8  */      {T1D,                 T2D}
   };
 
+Assembler::SIMD_RegVariant Assembler::_esize2regvariant[9] = {
+  INVALID,
+  B,
+  H,
+  INVALID,
+  S,
+  INVALID,
+  INVALID,
+  INVALID,
+  D,
+};
 
-Assembler::SIMD_Arrangement Assembler::esize2arrangement(int esize, bool isQ) {
-    guarantee(esize == 1 || esize == 2 || esize == 4 || esize == 8, "unsupported element size");
-    return _esize2arrangement_table[esize][isQ];
+Assembler::SIMD_Arrangement Assembler::esize2arrangement(unsigned esize, bool isQ) {
+  guarantee(esize < ARRAY_SIZE(_esize2arrangement_table) &&
+         _esize2arrangement_table[esize][isQ] != INVALID_ARRANGEMENT, "unsupported element size");
+  return _esize2arrangement_table[esize][isQ];
+}
+
+Assembler::SIMD_RegVariant Assembler::elemBytes_to_regVariant(unsigned esize) {
+  guarantee(esize < ARRAY_SIZE(_esize2regvariant) && _esize2regvariant[esize] != INVALID,
+         "unsupported element size");
+  return _esize2regvariant[esize];
+}
+
+Assembler::SIMD_RegVariant Assembler::elemType_to_regVariant(BasicType bt) {
+  return elemBytes_to_regVariant(type2aelembytes(bt));
 }
 
 void Assembler::emit_data64(jlong data,

src/hotspot/cpu/aarch64/assembler_aarch64.hpp

@@ -647,22 +647,22 @@ typedef enum {
 
 class Assembler : public AbstractAssembler {
 
+public:
+
 #ifndef PRODUCT
   static const uintptr_t asm_bp;
 
-  void emit_long(jint x) {
+  void emit_int32(jint x) {
     if ((uintptr_t)pc() == asm_bp)
       NOP();
     AbstractAssembler::emit_int32(x);
   }
 #else
-  void emit_long(jint x) {
+  void emit_int32(jint x) {
     AbstractAssembler::emit_int32(x);
   }
 #endif
 
-public:
-
   enum { instruction_size = 4 };
 
   //---<  calculate length of instruction  >---
@@ -1502,17 +1502,20 @@ public:
     T8B, T16B, T4H, T8H, T2S, T4S, T1D, T2D, T1Q, INVALID_ARRANGEMENT
   };
 
+  enum SIMD_RegVariant {
+      B, H, S, D, Q, INVALID
+  };
+
 private:
 
   static SIMD_Arrangement _esize2arrangement_table[9][2];
+  static SIMD_RegVariant _esize2regvariant[9];
 
 public:
 
-  static SIMD_Arrangement esize2arrangement(int esize, bool isQ);
-
-  enum SIMD_RegVariant {
-    B, H, S, D, Q, INVALID
-  };
+  static SIMD_Arrangement esize2arrangement(unsigned esize, bool isQ);
+  static SIMD_RegVariant elemType_to_regVariant(BasicType bt);
+  static SIMD_RegVariant elemBytes_to_regVariant(unsigned esize);
 
   enum shift_kind { LSL, LSR, ASR, ROR };
 
@@ -2927,7 +2930,7 @@ public:
     f(0, 10), rf(Vn, 5), rf(Vd, 0);
   }
 
-// SVE arithmetics - unpredicated
+// SVE arithmetic - unpredicated
 #define INSN(NAME, opcode)                                                             \
   void NAME(FloatRegister Zd, SIMD_RegVariant T, FloatRegister Zn, FloatRegister Zm) { \
     starti;                                                                            \
@@ -2964,7 +2967,7 @@ private:
 
 public:
 
-// SVE integer arithmetics - predicate
+// SVE integer arithmetic - predicate
 #define INSN(NAME, op1, op2)                                                                            \
   void NAME(FloatRegister Zdn_or_Zd_or_Vd, SIMD_RegVariant T, PRegister Pg, FloatRegister Znm_or_Vn) {  \
     assert(T != Q, "invalid register variant");                                                         \
@@ -2992,7 +2995,7 @@ public:
   INSN(sve_uaddv, 0b00000100, 0b000001001); // unsigned add reduction to scalar
 #undef INSN
 
-// SVE floating-point arithmetics - predicate
+// SVE floating-point arithmetic - predicate
 #define INSN(NAME, op1, op2)                                                                          \
   void NAME(FloatRegister Zd_or_Zdn_or_Vd, SIMD_RegVariant T, PRegister Pg, FloatRegister Zn_or_Zm) { \
     assert(T == S || T == D, "invalid register variant");                                             \
@@ -3121,7 +3124,7 @@ private:
 
 public:
 
-// SVE load/store - predicated
+// SVE contiguous load/store
 #define INSN(NAME, op1, type, imm_op2, scalar_op2)                                   \
   void NAME(FloatRegister Zt, SIMD_RegVariant T, PRegister Pg, const Address &a) {   \
     assert(T != Q, "invalid register variant");                                      \
@@ -3138,6 +3141,23 @@ public:
   INSN(sve_st1d, 0b1110010, 0b11, 0b111, 0b010);
 #undef INSN
 
+// Gather/scatter load/store (SVE) - scalar plus vector
+#define INSN(NAME, op1, type, op2, op3)                                         \
+  void NAME(FloatRegister Zt, PRegister Pg, Register Xn, FloatRegister Zm) {    \
+    starti;                                                                     \
+    f(op1, 31, 25), f(type, 24, 23), f(op2, 22, 21), rf(Zm, 16);                \
+    f(op3, 15, 13), pgrf(Pg, 10), srf(Xn, 5), rf(Zt, 0);                        \
+  }
+  // SVE 32-bit gather load words (scalar plus 32-bit scaled offsets)
+  INSN(sve_ld1w_gather,  0b1000010, 0b10, 0b01, 0b010);
+  // SVE 64-bit gather load (scalar plus 32-bit unpacked scaled offsets)
+  INSN(sve_ld1d_gather,  0b1100010, 0b11, 0b01, 0b010);
+  // SVE 32-bit scatter store (scalar plus 32-bit scaled offsets)
+  INSN(sve_st1w_scatter, 0b1110010, 0b10, 0b11, 0b100);
+  // SVE 64-bit scatter store (scalar plus unpacked 32-bit scaled offsets)
+  INSN(sve_st1d_scatter, 0b1110010, 0b11, 0b01, 0b100);
+#undef INSN
+
 // SVE load/store - unpredicated
 #define INSN(NAME, op1)                                                         \
   void NAME(FloatRegister Zt, const Address &a)  {                              \
@@ -3151,6 +3171,7 @@ public:
   INSN(sve_str, 0b111); // STR (vector)
 #undef INSN
 
+// SVE stack frame adjustment
 #define INSN(NAME, op) \
   void NAME(Register Xd, Register Xn, int imm6) {                 \
     starti;                                                       \
@@ -3158,8 +3179,8 @@ public:
     srf(Xn, 16), f(0b01010, 15, 11), sf(imm6, 10, 5), srf(Xd, 0); \
   }
 
-  INSN(sve_addvl, 0b01);
-  INSN(sve_addpl, 0b11);
+  INSN(sve_addvl, 0b01); // Add multiple of vector register size to scalar register
+  INSN(sve_addpl, 0b11); // Add multiple of predicate register size to scalar register
 #undef INSN
 
 // SVE inc/dec register by element count
@@ -3175,15 +3196,15 @@ public:
   INSN(sve_dec, 1);
 #undef INSN
 
-  // SVE predicate count
-  void sve_cntp(Register Xd, SIMD_RegVariant T, PRegister Pg, PRegister Pn) {
+  // SVE increment register by predicate count
+  void sve_incp(const Register rd, SIMD_RegVariant T, PRegister pg) {
     starti;
     assert(T != Q, "invalid size");
-    f(0b00100101, 31, 24), f(T, 23, 22), f(0b10000010, 21, 14);
-    prf(Pg, 10), f(0, 9), prf(Pn, 5), rf(Xd, 0);
+    f(0b00100101, 31, 24), f(T, 23, 22), f(0b1011001000100, 21, 9),
+    prf(pg, 5), rf(rd, 0);
   }
 
-  // SVE dup scalar
+  // SVE broadcast general-purpose register to vector elements (unpredicated)
   void sve_dup(FloatRegister Zd, SIMD_RegVariant T, Register Rn) {
     starti;
     assert(T != Q, "invalid size");
@@ -3191,7 +3212,7 @@ public:
     srf(Rn, 5), rf(Zd, 0);
   }
 
-  // SVE dup imm
+  // SVE broadcast signed immediate to vector elements (unpredicated)
   void sve_dup(FloatRegister Zd, SIMD_RegVariant T, int imm8) {
     starti;
     assert(T != Q, "invalid size");
@@ -3214,19 +3235,119 @@ public:
     f(pattern, 9, 5), f(0b0, 4), prf(pd, 0);
   }
 
-// Integer comparisons (SVE)
-#define INSN(NAME, cond)                                                                          \
-  void NAME(PRegister Pd, SIMD_RegVariant T, PRegister Pg, FloatRegister Zn, FloatRegister Zm)  { \
-    starti;                                                                                       \
-    assert(T != Q, "invalid size");                                                               \
-    f(0b00100100, 31, 24), f(T, 23, 22), f(0, 21), rf(Zm, 16), f((cond >> 1) & 7, 15, 13);        \
-    pgrf(Pg, 10), rf(Zn, 5), f(cond & 1, 4), prf(Pd, 0);                                          \
+  // SVE copy general-purpose register to vector elements (predicated)
+  void sve_cpy(FloatRegister Zd, SIMD_RegVariant T, PRegister Pg, Register Rn) {
+    starti;
+    assert(T != Q, "invalid size");
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b101000101, 21, 13);
+    pgrf(Pg, 10), srf(Rn, 5), rf(Zd, 0);
   }
 
-  INSN(sve_cmpeq, 0b1010);  // Compare signed equal to vector
-  INSN(sve_cmpne, 0b1011);  // Compare not equal to vector
-  INSN(sve_cmpge, 0b1000);  // Compare signed greater than or equal to vector
-  INSN(sve_cmpgt, 0b1001);  // Compare signed greater than vector
+  // SVE copy signed integer immediate to vector elements (predicated)
+  void sve_cpy(FloatRegister Zd, SIMD_RegVariant T, PRegister Pg, int imm8, bool isMerge) {
+    starti;
+    assert(T != Q, "invalid size");
+    int sh = 0;
+    if (imm8 <= 127 && imm8 >= -128) {
+      sh = 0;
+    } else if (T != B && imm8 <= 32512 && imm8 >= -32768 && (imm8 & 0xff) == 0) {
+      sh = 1;
+      imm8 = (imm8 >> 8);
+    } else {
+      guarantee(false, "invalid immediate");
+    }
+    int m = isMerge ? 1 : 0;
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b01, 21, 20);
+    prf(Pg, 16), f(0b0, 15), f(m, 14), f(sh, 13), sf(imm8, 12, 5), rf(Zd, 0);
+  }
+
+  // SVE conditionally select elements from two vectors
+  void sve_sel(FloatRegister Zd, SIMD_RegVariant T, PRegister Pg,
+               FloatRegister Zn, FloatRegister Zm) {
+    starti;
+    assert(T != Q, "invalid size");
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b1, 21), rf(Zm, 16);
+    f(0b11, 15, 14), prf(Pg, 10), rf(Zn, 5), rf(Zd, 0);
+  }
+
+// SVE Integer/Floating-Point Compare - Vectors
+#define INSN(NAME, op1, op2, fp)  \
+  void NAME(Condition cond, PRegister Pd, SIMD_RegVariant T, PRegister Pg,             \
+            FloatRegister Zn, FloatRegister Zm) {                                      \
+    starti;                                                                            \
+    if (fp == 0) {                                                                     \
+      assert(T != Q, "invalid size");                                                  \
+    } else {                                                                           \
+      assert(T != B && T != Q, "invalid size");                                        \
+      assert(cond != HI && cond != HS, "invalid condition for fcm");                   \
+    }                                                                                  \
+    int cond_op;                                                                       \
+    switch(cond) {                                                                     \
+      case EQ: cond_op = (op2 << 2) | 0b10; break;                                     \
+      case NE: cond_op = (op2 << 2) | 0b11; break;                                     \
+      case GE: cond_op = (op2 << 2) | 0b00; break;                                     \
+      case GT: cond_op = (op2 << 2) | 0b01; break;                                     \
+      case HI: cond_op = 0b0001; break;                                                \
+      case HS: cond_op = 0b0000; break;                                                \
+      default:                                                                         \
+        ShouldNotReachHere();                                                          \
+    }                                                                                  \
+    f(op1, 31, 24), f(T, 23, 22), f(0, 21), rf(Zm, 16), f((cond_op >> 1) & 7, 15, 13); \
+    pgrf(Pg, 10), rf(Zn, 5), f(cond_op & 1, 4), prf(Pd, 0);                            \
+  }
+
+  INSN(sve_cmp, 0b00100100, 0b10, 0);
+  INSN(sve_fcm, 0b01100101, 0b01, 1);
+#undef INSN
+
+// SVE Integer Compare - Signed Immediate
+void sve_cmp(Condition cond, PRegister Pd, SIMD_RegVariant T,
+             PRegister Pg, FloatRegister Zn, int imm5) {
+  starti;
+  assert(T != Q, "invalid size");
+  guarantee(-16 <= imm5 && imm5 <= 15, "invalid immediate");
+  int cond_op;
+  switch(cond) {
+    case EQ: cond_op = 0b1000; break;
+    case NE: cond_op = 0b1001; break;
+    case GE: cond_op = 0b0000; break;
+    case GT: cond_op = 0b0001; break;
+    case LE: cond_op = 0b0011; break;
+    case LT: cond_op = 0b0010; break;
+    default:
+      ShouldNotReachHere();
+  }
+  f(0b00100101, 31, 24), f(T, 23, 22), f(0b0, 21), sf(imm5, 20, 16),
+  f((cond_op >> 1) & 0x7, 15, 13), pgrf(Pg, 10), rf(Zn, 5);
+  f(cond_op & 0x1, 4), prf(Pd, 0);
+}
+
+// SVE unpack vector elements
+#define INSN(NAME, op) \
+  void NAME(FloatRegister Zd, SIMD_RegVariant T, FloatRegister Zn) { \
+    starti;                                                          \
+    assert(T != B && T != Q, "invalid size");                        \
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b1100, 21, 18);          \
+    f(op, 17, 16), f(0b001110, 15, 10), rf(Zn, 5), rf(Zd, 0);        \
+  }
+
+  INSN(sve_uunpkhi, 0b11); // Signed unpack and extend half of vector - high half
+  INSN(sve_uunpklo, 0b10); // Signed unpack and extend half of vector - low half
+  INSN(sve_sunpkhi, 0b01); // Unsigned unpack and extend half of vector - high half
+  INSN(sve_sunpklo, 0b00); // Unsigned unpack and extend half of vector - low half
+#undef INSN
+
+// SVE permute vector elements
+#define INSN(NAME, op) \
+  void NAME(FloatRegister Zd, SIMD_RegVariant T, FloatRegister Zn, FloatRegister Zm) { \
+    starti;                                                                            \
+    assert(T != Q, "invalid size");                                                    \
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b1, 21), rf(Zm, 16);                       \
+    f(0b01101, 15, 11), f(op, 10), rf(Zn, 5), rf(Zd, 0);                               \
+  }
+
+  INSN(sve_uzp1, 0b0); // Concatenate even elements from two vectors
+  INSN(sve_uzp2, 0b1); // Concatenate odd elements from two vectors
 #undef INSN
 
 // Predicate counted loop (SVE) (32-bit variants are not included)
@@ -3245,15 +3366,26 @@ public:
   INSN(sve_whilels, 0b111);  // While incrementing unsigned scalar lower than or the same as scalar
 #undef INSN
 
-  // Predicate scan (SVE)
-
-  // Break after the first true condition
-  void sve_brka(PRegister pd, PRegister pg, PRegister pn, bool isMerge) {
+  // SVE predicate reverse
+  void sve_rev(PRegister Pd, SIMD_RegVariant T, PRegister Pn) {
     starti;
-    f(0b00100101, 31, 24), f(0b00, 23, 22), f(0b01000001, 21, 14),
-    prf(pg, 10), f(0b0, 9), prf(pn, 5), f(isMerge ? 1 : 0, 4), prf(pd, 0);
+    assert(T != Q, "invalid size");
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b1101000100000, 21, 9);
+    prf(Pn, 5), f(0, 4), prf(Pd, 0);
   }
 
+// SVE partition break condition
+#define INSN(NAME, op) \
+  void NAME(PRegister Pd, PRegister Pg, PRegister Pn, bool isMerge) {      \
+    starti;                                                                \
+    f(0b00100101, 31, 24), f(op, 23, 22), f(0b01000001, 21, 14);           \
+    prf(Pg, 10), f(0b0, 9), prf(Pn, 5), f(isMerge ? 1 : 0, 4), prf(Pd, 0); \
+  }
+
+  INSN(sve_brka, 0b00); // Break after first true condition
+  INSN(sve_brkb, 0b10); // Break before first true condition
+#undef INSN
+
 // Element count and increment scalar (SVE)
 #define INSN(NAME, TYPE)                                                             \
   void NAME(Register Xdn, unsigned imm4 = 1, int pattern = 0b11111) {                \
@@ -3268,14 +3400,122 @@ public:
   INSN(sve_cntd, D);  // Set scalar to multiple of 64-bit predicate constraint element count
 #undef INSN
 
-  // Predicate count and increment scalar (SVE)
-
-  // Set scalar to the number of Active predicate elements that are TRUE
-  void sve_incp(const Register rd, SIMD_RegVariant T, PRegister pg) {
+  // Set scalar to active predicate element count
+  void sve_cntp(Register Xd, SIMD_RegVariant T, PRegister Pg, PRegister Pn) {
     starti;
     assert(T != Q, "invalid size");
-    f(0b00100101, 31, 24), f(T, 23, 22), f(0b1011001000100, 21, 9),
-    prf(pg, 5), rf(rd, 0);
+    f(0b00100101, 31, 24), f(T, 23, 22), f(0b10000010, 21, 14);
+    prf(Pg, 10), f(0, 9), prf(Pn, 5), rf(Xd, 0);
+  }
+
+  // SVE convert signed integer to floating-point (predicated)
+  void sve_scvtf(FloatRegister Zd, SIMD_RegVariant T_dst, PRegister Pg,
+                 FloatRegister Zn, SIMD_RegVariant T_src) {
+    starti;
+    assert(T_src != B && T_dst != B && T_src != Q && T_dst != Q &&
+           (T_src != H || T_dst == T_src), "invalid register variant");
+    int opc = T_dst;
+    int opc2 = T_src;
+    // In most cases we can treat T_dst, T_src as opc, opc2,
+    // except for the following two combinations.
+    // +-----+------+---+------------------------------------+
+    // | opc | opc2 | U |        Instruction Details         |
+    // +-----+------+---+------------------------------------+
+    // |  11 |   00 | 0 | SCVTF - 32-bit to double-precision |
+    // |  11 |   10 | 0 | SCVTF - 64-bit to single-precision |
+    // +-----+------+---+------------------------------------+
+    if (T_src == S && T_dst == D) {
+      opc = 0b11;
+      opc2 = 0b00;
+    } else if (T_src == D && T_dst == S) {
+      opc = 0b11;
+      opc2 = 0b10;
+    }
+    f(0b01100101, 31, 24), f(opc, 23, 22), f(0b010, 21, 19);
+    f(opc2, 18, 17), f(0b0101, 16, 13);
+    pgrf(Pg, 10), rf(Zn, 5), rf(Zd, 0);
+  }
+
+  // SVE floating-point convert to signed integer, rounding toward zero (predicated)
+  void sve_fcvtzs(FloatRegister Zd, SIMD_RegVariant T_dst, PRegister Pg,
+                  FloatRegister Zn, SIMD_RegVariant T_src) {
+    starti;
+    assert(T_src != B && T_dst != B && T_src != Q && T_dst != Q &&
+           (T_dst != H || T_src == H), "invalid register variant");
+    int opc = T_src;
+    int opc2 = T_dst;
+    // In most cases we can treat T_src, T_dst as opc, opc2,
+    // except for the following two combinations.
+    // +-----+------+---+-------------------------------------+
+    // | opc | opc2 | U |         Instruction Details         |
+    // +-----+------+---+-------------------------------------+
+    // |  11 |  10  | 0 | FCVTZS - single-precision to 64-bit |
+    // |  11 |  00  | 0 | FCVTZS - double-precision to 32-bit |
+    // +-----+------+---+-------------------------------------+
+    if (T_src == S && T_dst == D) {
+      opc = 0b11;
+      opc2 = 0b10;
+    } else if (T_src == D && T_dst == S) {
+      opc = 0b11;
+      opc2 = 0b00;
+    }
+    f(0b01100101, 31, 24), f(opc, 23, 22), f(0b011, 21, 19);
+    f(opc2, 18, 17), f(0b0101, 16, 13);
+    pgrf(Pg, 10), rf(Zn, 5), rf(Zd, 0);
+  }
+
+  // SVE floating-point convert precision (predicated)
+  void sve_fcvt(FloatRegister Zd, SIMD_RegVariant T_dst, PRegister Pg,
+                FloatRegister Zn, SIMD_RegVariant T_src) {
+    starti;
+    assert(T_src != B && T_dst != B && T_src != Q && T_dst != Q &&
+           T_src != T_dst, "invalid register variant");
+    guarantee(T_src != H && T_dst != H, "half-precision unsupported");
+    f(0b01100101, 31, 24), f(0b11, 23, 22), f(0b0010, 21, 18);
+    f(T_dst, 17, 16), f(0b101, 15, 13);
+    pgrf(Pg, 10), rf(Zn, 5), rf(Zd, 0);
+  }
+
+// SVE extract element to general-purpose register
+#define INSN(NAME, before)                                                      \
+  void NAME(Register Rd, SIMD_RegVariant T, PRegister Pg,  FloatRegister Zn) {  \
+    starti;                                                                     \
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b10000, 21, 17);                    \
+    f(before, 16), f(0b101, 15, 13);                                            \
+    pgrf(Pg, 10), rf(Zn, 5), rf(Rd, 0);                                         \
+  }
+
+  INSN(sve_lasta, 0b0);
+  INSN(sve_lastb, 0b1);
+#undef INSN
+
+// SVE extract element to SIMD&FP scalar register
+#define INSN(NAME, before)                                                           \
+  void NAME(FloatRegister Vd, SIMD_RegVariant T, PRegister Pg,  FloatRegister Zn) {  \
+    starti;                                                                          \
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b10001, 21, 17);                         \
+    f(before, 16), f(0b100, 15, 13);                                                 \
+    pgrf(Pg, 10), rf(Zn, 5), rf(Vd, 0);                                              \
+  }
+
+  INSN(sve_lasta, 0b0);
+  INSN(sve_lastb, 0b1);
+#undef INSN
+
+  // SVE create index starting from and incremented by immediate
+  void sve_index(FloatRegister Zd, SIMD_RegVariant T, int imm1, int imm2) {
+    starti;
+    f(0b00000100, 31, 24), f(T, 23, 22), f(0b1, 21);
+    sf(imm2, 20, 16), f(0b010000, 15, 10);
+    sf(imm1, 9, 5), rf(Zd, 0);
+  }
+
+  // SVE programmable table lookup/permute using vector of element indices
+  void sve_tbl(FloatRegister Zd, SIMD_RegVariant T, FloatRegister Zn, FloatRegister Zm) {
+    starti;
+    assert(T != Q, "invalid size");
+    f(0b00000101, 31, 24), f(T, 23, 22), f(0b1, 21), rf(Zm, 16);
+    f(0b001100, 15, 10), rf(Zn, 5), rf(Zd, 0);
   }
 
   Assembler(CodeBuffer* code) : AbstractAssembler(code) {

src/hotspot/cpu/aarch64/atomic_aarch64.hpp

@@ -37,6 +37,8 @@ typedef uint64_t (*aarch64_atomic_stub_t)(volatile void *ptr, uint64_t arg1, uin
 // Pointers to stubs
 extern aarch64_atomic_stub_t aarch64_atomic_fetch_add_4_impl;
 extern aarch64_atomic_stub_t aarch64_atomic_fetch_add_8_impl;
+extern aarch64_atomic_stub_t aarch64_atomic_fetch_add_4_relaxed_impl;
+extern aarch64_atomic_stub_t aarch64_atomic_fetch_add_8_relaxed_impl;
 extern aarch64_atomic_stub_t aarch64_atomic_xchg_4_impl;
 extern aarch64_atomic_stub_t aarch64_atomic_xchg_8_impl;
 extern aarch64_atomic_stub_t aarch64_atomic_cmpxchg_1_impl;

src/hotspot/cpu/aarch64/c2_MacroAssembler_aarch64.cpp

@@ -584,7 +584,7 @@ void C2_MacroAssembler::string_indexof_char_sve(Register str1, Register cnt1,
 
     // Perform the comparison. An element of the destination predicate is set
     // to active if the particular char is matched.
-    sve_cmpeq(tmp_pdn, T, tmp_pg, ztmp1, ztmp2);
+    sve_cmp(Assembler::EQ, tmp_pdn, T, tmp_pg, ztmp1, ztmp2);
 
     // Branch if the particular char is found.
     br(NE, MATCH);
@@ -905,7 +905,7 @@ void C2_MacroAssembler::string_compare(Register str1, Register str2,
 
 void C2_MacroAssembler::neon_compare(FloatRegister dst, BasicType bt, FloatRegister src1,
                                      FloatRegister src2, int cond, bool isQ) {
-  SIMD_Arrangement size = esize2arrangement(type2aelembytes(bt), isQ);
+  SIMD_Arrangement size = esize2arrangement((unsigned)type2aelembytes(bt), isQ);
   if (bt == T_FLOAT || bt == T_DOUBLE) {
     switch (cond) {
       case BoolTest::eq: fcmeq(dst, size, src1, src2); break;
@@ -944,3 +944,56 @@ void C2_MacroAssembler::neon_compare(FloatRegister dst, BasicType bt, FloatRegis
     }
   }
 }
+
+void C2_MacroAssembler::sve_compare(PRegister pd, BasicType bt, PRegister pg,
+                                    FloatRegister zn, FloatRegister zm, int cond) {
+  assert(pg->is_governing(), "This register has to be a governing predicate register");
+  FloatRegister z1 = zn, z2 = zm;
+  // Convert the original BoolTest condition to Assembler::condition.
+  Condition condition;
+  switch (cond) {
+    case BoolTest::eq: condition = Assembler::EQ; break;
+    case BoolTest::ne: condition = Assembler::NE; break;
+    case BoolTest::le: z1 = zm; z2 = zn; condition = Assembler::GE; break;
+    case BoolTest::ge: condition = Assembler::GE; break;
+    case BoolTest::lt: z1 = zm; z2 = zn; condition = Assembler::GT; break;
+    case BoolTest::gt: condition = Assembler::GT; break;
+    default:
+      assert(false, "unsupported compare condition");
+      ShouldNotReachHere();
+  }
+
+  SIMD_RegVariant size = elemType_to_regVariant(bt);
+  if (bt == T_FLOAT || bt == T_DOUBLE) {
+    sve_fcm(condition, pd, size, pg, z1, z2);
+  } else {
+    assert(is_integral_type(bt), "unsupported element type");
+    sve_cmp(condition, pd, size, pg, z1, z2);
+  }
+}
+
+void C2_MacroAssembler::sve_vmask_reduction(int opc, Register dst, SIMD_RegVariant size, FloatRegister src,
+                                            PRegister pg, PRegister pn, int length) {
+  assert(pg->is_governing(), "This register has to be a governing predicate register");
+  // The conditional flags will be clobbered by this function
+  sve_cmp(Assembler::NE, pn, size, pg, src, 0);
+  switch (opc) {
+    case Op_VectorMaskTrueCount:
+      sve_cntp(dst, size, ptrue, pn);
+      break;
+    case Op_VectorMaskFirstTrue:
+      sve_brkb(pn, pg, pn, false);
+      sve_cntp(dst, size, ptrue, pn);
+      break;
+    case Op_VectorMaskLastTrue:
+      sve_rev(pn, size, pn);
+      sve_brkb(pn, ptrue, pn, false);
+      sve_cntp(dst, size, ptrue, pn);
+      movw(rscratch1, length - 1);
+      subw(dst, rscratch1, dst);
+      break;
+    default:
+      assert(false, "unsupported");
+      ShouldNotReachHere();
+  }
+}

src/hotspot/cpu/aarch64/c2_MacroAssembler_aarch64.hpp

@@ -58,4 +58,30 @@
   void neon_compare(FloatRegister dst, BasicType bt, FloatRegister src1,
                     FloatRegister src2, int cond, bool isQ);
 
+  void sve_compare(PRegister pd, BasicType bt, PRegister pg,
+                   FloatRegister zn, FloatRegister zm, int cond);
+
+  void sve_vmask_reduction(int opc, Register dst, SIMD_RegVariant size, FloatRegister src,
+                           PRegister pg, PRegister pn, int length = MaxVectorSize);
+
+  // Generate predicate through whilelo, by comparing ZR with an unsigned
+  // immediate. rscratch1 will be clobbered.
+  inline void sve_whilelo_zr_imm(PRegister pd, SIMD_RegVariant size, uint imm) {
+    assert(UseSVE > 0, "not supported");
+    mov(rscratch1, imm);
+    sve_whilelo(pd, size, zr, rscratch1);
+  }
+
+  // Extract a scalar element from an sve vector at position 'idx'.
+  // rscratch1 will be clobbered.
+  // T could be FloatRegister or Register.
+  template<class T>
+  inline void sve_extract(T dst, SIMD_RegVariant size, PRegister pg, FloatRegister src, int idx) {
+    assert(UseSVE > 0, "not supported");
+    assert(pg->is_governing(), "This register has to be a governing predicate register");
+    mov(rscratch1, idx);
+    sve_whilele(pg, size, zr, rscratch1);
+    sve_lastb(dst, size, pg, src);
+  }
+
 #endif // CPU_AARCH64_C2_MACROASSEMBLER_AARCH64_HPP

src/hotspot/cpu/aarch64/frame_aarch64.cpp

@@ -814,7 +814,6 @@ frame::frame(void* sp, void* fp, void* pc) {
   init((intptr_t*)sp, (intptr_t*)fp, (address)pc);
 }
 
-void frame::pd_ps() {}
 #endif
 
 void JavaFrameAnchor::make_walkable(JavaThread* thread) {

src/hotspot/cpu/aarch64/gc/shared/barrierSetAssembler_aarch64.cpp

@@ -276,7 +276,7 @@ void BarrierSetAssembler::c2i_entry_barrier(MacroAssembler* masm) {
   __ load_method_holder_cld(rscratch1, rmethod);
 
   // Is it a strong CLD?
-  __ ldr(rscratch2, Address(rscratch1, ClassLoaderData::keep_alive_offset()));
+  __ ldrw(rscratch2, Address(rscratch1, ClassLoaderData::keep_alive_offset()));
   __ cbnz(rscratch2, method_live);
 
   // Is it a weak but alive CLD?

src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp

@@ -2537,7 +2537,9 @@ void MacroAssembler::pop_CPU_state(bool restore_vectors, bool use_sve,
           as_FloatRegister(i+3), restore_vectors ? T2D : T1D, Address(post(sp, step)));
   }
 
-  if (restore_vectors) {
+  // We may use predicate registers and rely on ptrue with SVE,
+  // regardless of wide vector (> 8 bytes) used or not.
+  if (use_sve) {
     reinitialize_ptrue();
   }
 
@@ -4773,23 +4775,37 @@ void MacroAssembler::fill_words(Register base, Register cnt, Register value)
 {
 //  Algorithm:
 //
-//    scratch1 = cnt & 7;
+//    if (cnt == 0) {
+//      return;
+//    }
+//    if ((p & 8) != 0) {
+//      *p++ = v;
+//    }
+//
+//    scratch1 = cnt & 14;
 //    cnt -= scratch1;
 //    p += scratch1;
-//    switch (scratch1) {
+//    switch (scratch1 / 2) {
 //      do {
-//        cnt -= 8;
-//          p[-8] = v;
+//        cnt -= 16;
+//          p[-16] = v;
+//          p[-15] = v;
 //        case 7:
-//          p[-7] = v;
+//          p[-14] = v;
+//          p[-13] = v;
 //        case 6:
-//          p[-6] = v;
+//          p[-12] = v;
+//          p[-11] = v;
 //          // ...
 //        case 1:
+//          p[-2] = v;
 //          p[-1] = v;
 //        case 0:
-//          p += 8;
+//          p += 16;
 //      } while (cnt);
+//    }
+//    if ((cnt & 1) == 1) {
+//      *p++ = v;
 //    }
 
   assert_different_registers(base, cnt, value, rscratch1, rscratch2);

src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp

@@ -1296,11 +1296,37 @@ public:
   void kernel_crc32c_using_crc32c(Register crc, Register buf,
         Register len, Register tmp0, Register tmp1, Register tmp2,
         Register tmp3);
+
+  void ghash_modmul (FloatRegister result,
+                     FloatRegister result_lo, FloatRegister result_hi, FloatRegister b,
+                     FloatRegister a, FloatRegister vzr, FloatRegister a1_xor_a0, FloatRegister p,
+                     FloatRegister t1, FloatRegister t2, FloatRegister t3);
+  void ghash_load_wide(int index, Register data, FloatRegister result, FloatRegister state);
 public:
   void multiply_to_len(Register x, Register xlen, Register y, Register ylen, Register z,
                        Register zlen, Register tmp1, Register tmp2, Register tmp3,
                        Register tmp4, Register tmp5, Register tmp6, Register tmp7);
   void mul_add(Register out, Register in, Register offs, Register len, Register k);
+  void ghash_multiply(FloatRegister result_lo, FloatRegister result_hi,
+                      FloatRegister a, FloatRegister b, FloatRegister a1_xor_a0,
+                      FloatRegister tmp1, FloatRegister tmp2, FloatRegister tmp3);
+  void ghash_multiply_wide(int index,
+                           FloatRegister result_lo, FloatRegister result_hi,
+                           FloatRegister a, FloatRegister b, FloatRegister a1_xor_a0,
+                           FloatRegister tmp1, FloatRegister tmp2, FloatRegister tmp3);
+  void ghash_reduce(FloatRegister result, FloatRegister lo, FloatRegister hi,
+                    FloatRegister p, FloatRegister z, FloatRegister t1);
+  void ghash_reduce_wide(int index, FloatRegister result, FloatRegister lo, FloatRegister hi,
+                    FloatRegister p, FloatRegister z, FloatRegister t1);
+  void ghash_processBlocks_wide(address p, Register state, Register subkeyH,
+                                Register data, Register blocks, int unrolls);
+
+
+  void aesenc_loadkeys(Register key, Register keylen);
+  void aesecb_encrypt(Register from, Register to, Register keylen,
+                      FloatRegister data = v0, int unrolls = 1);
+  void aesecb_decrypt(Register from, Register to, Register key, Register keylen);
+  void aes_round(FloatRegister input, FloatRegister subkey);
 
   // Place an ISB after code may have been modified due to a safepoint.
   void safepoint_isb();

src/hotspot/cpu/aarch64/macroAssembler_aarch64_aes.cpp

@@ -0,0 +1,680 @@
+/*
+ * Copyright (c) 2003, 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2014, 2021, Red Hat Inc. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ *
+ */
+
+#include "precompiled.hpp"
+
+#include "asm/assembler.hpp"
+#include "asm/assembler.inline.hpp"
+#include "macroAssembler_aarch64.hpp"
+#include "memory/resourceArea.hpp"
+#include "runtime/stubRoutines.hpp"
+
+void MacroAssembler::aesecb_decrypt(Register from, Register to, Register key, Register keylen) {
+  Label L_doLast;
+
+  ld1(v0, T16B, from); // get 16 bytes of input
+
+  ld1(v5, T16B, post(key, 16));
+  rev32(v5, T16B, v5);
+
+  ld1(v1, v2, v3, v4, T16B, post(key, 64));
+  rev32(v1, T16B, v1);
+  rev32(v2, T16B, v2);
+  rev32(v3, T16B, v3);
+  rev32(v4, T16B, v4);
+  aesd(v0, v1);
+  aesimc(v0, v0);
+  aesd(v0, v2);
+  aesimc(v0, v0);
+  aesd(v0, v3);
+  aesimc(v0, v0);
+  aesd(v0, v4);
+  aesimc(v0, v0);
+
+  ld1(v1, v2, v3, v4, T16B, post(key, 64));
+  rev32(v1, T16B, v1);
+  rev32(v2, T16B, v2);
+  rev32(v3, T16B, v3);
+  rev32(v4, T16B, v4);
+  aesd(v0, v1);
+  aesimc(v0, v0);
+  aesd(v0, v2);
+  aesimc(v0, v0);
+  aesd(v0, v3);
+  aesimc(v0, v0);
+  aesd(v0, v4);
+  aesimc(v0, v0);
+
+  ld1(v1, v2, T16B, post(key, 32));
+  rev32(v1, T16B, v1);
+  rev32(v2, T16B, v2);
+
+  cmpw(keylen, 44);
+  br(Assembler::EQ, L_doLast);
+
+  aesd(v0, v1);
+  aesimc(v0, v0);
+  aesd(v0, v2);
+  aesimc(v0, v0);
+
+  ld1(v1, v2, T16B, post(key, 32));
+  rev32(v1, T16B, v1);
+  rev32(v2, T16B, v2);
+
+  cmpw(keylen, 52);
+  br(Assembler::EQ, L_doLast);
+
+  aesd(v0, v1);
+  aesimc(v0, v0);
+  aesd(v0, v2);
+  aesimc(v0, v0);
+
+  ld1(v1, v2, T16B, post(key, 32));
+  rev32(v1, T16B, v1);
+  rev32(v2, T16B, v2);
+
+  bind(L_doLast);
+
+  aesd(v0, v1);
+  aesimc(v0, v0);
+  aesd(v0, v2);
+
+  eor(v0, T16B, v0, v5);
+
+  st1(v0, T16B, to);
+
+  // Preserve the address of the start of the key
+  sub(key, key, keylen, LSL, exact_log2(sizeof (jint)));
+}
+
+// Load expanded key into v17..v31
+void MacroAssembler::aesenc_loadkeys(Register key, Register keylen) {
+  Label L_loadkeys_44, L_loadkeys_52;
+  cmpw(keylen, 52);
+  br(Assembler::LO, L_loadkeys_44);
+  br(Assembler::EQ, L_loadkeys_52);
+
+  ld1(v17, v18,  T16B,  post(key, 32));
+  rev32(v17,  T16B, v17);
+  rev32(v18,  T16B, v18);
+  bind(L_loadkeys_52);
+  ld1(v19, v20,  T16B,  post(key, 32));
+  rev32(v19,  T16B, v19);
+  rev32(v20,  T16B, v20);
+  bind(L_loadkeys_44);
+  ld1(v21, v22, v23, v24,  T16B,  post(key, 64));
+  rev32(v21,  T16B, v21);
+  rev32(v22,  T16B, v22);
+  rev32(v23,  T16B, v23);
+  rev32(v24,  T16B, v24);
+  ld1(v25, v26, v27, v28,  T16B,  post(key, 64));
+  rev32(v25,  T16B, v25);
+  rev32(v26,  T16B, v26);
+  rev32(v27,  T16B, v27);
+  rev32(v28,  T16B, v28);
+  ld1(v29, v30, v31,  T16B, post(key, 48));
+  rev32(v29,  T16B, v29);
+  rev32(v30,  T16B, v30);
+  rev32(v31,  T16B, v31);
+
+  // Preserve the address of the start of the key
+  sub(key, key, keylen, LSL, exact_log2(sizeof (jint)));
+}
+
+// NeoverseTM N1Software Optimization Guide:
+// Adjacent AESE/AESMC instruction pairs and adjacent AESD/AESIMC
+// instruction pairs will exhibit the performance characteristics
+// described in Section 4.6.
+void MacroAssembler::aes_round(FloatRegister input, FloatRegister subkey) {
+  aese(input, subkey); aesmc(input, input);
+}
+
+// KernelGenerator
+//
+// The abstract base class of an unrolled function generator.
+// Subclasses override generate(), length(), and next() to generate
+// unrolled and interleaved functions.
+//
+// The core idea is that a subclass defines a method which generates
+// the base case of a function and a method to generate a clone of it,
+// shifted to a different set of registers. KernelGenerator will then
+// generate several interleaved copies of the function, with each one
+// using a different set of registers.
+
+// The subclass must implement three methods: length(), which is the
+// number of instruction bundles in the intrinsic, generate(int n)
+// which emits the nth instruction bundle in the intrinsic, and next()
+// which takes an instance of the generator and returns a version of it,
+// shifted to a new set of registers.
+
+class KernelGenerator: public MacroAssembler {
+protected:
+  const int _unrolls;
+public:
+  KernelGenerator(Assembler *as, int unrolls)
+    : MacroAssembler(as->code()), _unrolls(unrolls) { }
+  virtual void generate(int index) = 0;
+  virtual int length() = 0;
+  virtual KernelGenerator *next() = 0;
+  int unrolls() { return _unrolls; }
+  void unroll();
+};
+
+void KernelGenerator::unroll() {
+  ResourceMark rm;
+  KernelGenerator **generators
+    = NEW_RESOURCE_ARRAY(KernelGenerator *, unrolls());
+
+  generators[0] = this;
+  for (int i = 1; i < unrolls(); i++) {
+    generators[i] = generators[i-1]->next();
+  }
+
+  for (int j = 0; j < length(); j++) {
+    for (int i = 0; i < unrolls(); i++) {
+      generators[i]->generate(j);
+    }
+  }
+}
+
+// An unrolled and interleaved generator for AES encryption.
+class AESKernelGenerator: public KernelGenerator {
+  Register _from, _to;
+  const Register _keylen;
+  FloatRegister _data;
+  const FloatRegister _subkeys;
+  bool _once;
+  Label _rounds_44, _rounds_52;
+
+public:
+  AESKernelGenerator(Assembler *as, int unrolls,
+                     Register from, Register to, Register keylen, FloatRegister data,
+                     FloatRegister subkeys, bool once = true)
+    : KernelGenerator(as, unrolls),
+      _from(from), _to(to), _keylen(keylen), _data(data),
+      _subkeys(subkeys), _once(once) {
+  }
+
+  virtual void generate(int index) {
+    switch (index) {
+    case  0:
+      if (_from != noreg) {
+        ld1(_data, T16B, _from); // get 16 bytes of input
+      }
+      break;
+    case  1:
+      if (_once) {
+        cmpw(_keylen, 52);
+        br(Assembler::LO, _rounds_44);
+        br(Assembler::EQ, _rounds_52);
+      }
+      break;
+    case  2:  aes_round(_data, _subkeys +  0);  break;
+    case  3:  aes_round(_data, _subkeys +  1);  break;
+    case  4:
+      if (_once)  bind(_rounds_52);
+      break;
+    case  5:  aes_round(_data, _subkeys +  2);  break;
+    case  6:  aes_round(_data, _subkeys +  3);  break;
+    case  7:
+      if (_once)  bind(_rounds_44);
+      break;
+    case  8:  aes_round(_data, _subkeys +  4);  break;
+    case  9:  aes_round(_data, _subkeys +  5);  break;
+    case 10:  aes_round(_data, _subkeys +  6);  break;
+    case 11:  aes_round(_data, _subkeys +  7);  break;
+    case 12:  aes_round(_data, _subkeys +  8);  break;
+    case 13:  aes_round(_data, _subkeys +  9);  break;
+    case 14:  aes_round(_data, _subkeys + 10);  break;
+    case 15:  aes_round(_data, _subkeys + 11);  break;
+    case 16:  aes_round(_data, _subkeys + 12);  break;
+    case 17:  aese(_data, _subkeys + 13);  break;
+    case 18:  eor(_data, T16B, _data, _subkeys + 14);  break;
+    case 19:
+      if (_to != noreg) {
+        st1(_data, T16B, _to);
+      }
+      break;
+    default: ShouldNotReachHere();
+    }
+  }
+
+  virtual KernelGenerator *next() {
+    return new AESKernelGenerator(this, _unrolls,
+                                  _from, _to, _keylen,
+                                  _data + 1, _subkeys, /*once*/false);
+  }
+
+  virtual int length() { return 20; }
+};
+
+// Uses expanded key in v17..v31
+// Returns encrypted values in inputs.
+// If to != noreg, store value at to; likewise from
+// Preserves key, keylen
+// Increments from, to
+// Input data in v0, v1, ...
+// unrolls controls the number of times to unroll the generated function
+void MacroAssembler::aesecb_encrypt(Register from, Register to, Register keylen,
+                                    FloatRegister data, int unrolls) {
+  AESKernelGenerator(this, unrolls, from, to, keylen, data, v17) .unroll();
+}
+
+// ghash_multiply and ghash_reduce are the non-unrolled versions of
+// the GHASH function generators.
+void MacroAssembler::ghash_multiply(FloatRegister result_lo, FloatRegister result_hi,
+                                     FloatRegister a, FloatRegister b, FloatRegister a1_xor_a0,
+                                     FloatRegister tmp1, FloatRegister tmp2, FloatRegister tmp3) {
+  // Karatsuba multiplication performs a 128*128 -> 256-bit
+  // multiplication in three 128-bit multiplications and a few
+  // additions.
+  //
+  // (C1:C0) = A1*B1, (D1:D0) = A0*B0, (E1:E0) = (A0+A1)(B0+B1)
+  // (A1:A0)(B1:B0) = C1:(C0+C1+D1+E1):(D1+C0+D0+E0):D0
+  //
+  // Inputs:
+  //
+  // A0 in a.d[0]     (subkey)
+  // A1 in a.d[1]
+  // (A1+A0) in a1_xor_a0.d[0]
+  //
+  // B0 in b.d[0]     (state)
+  // B1 in b.d[1]
+
+  ext(tmp1, T16B, b, b, 0x08);
+  pmull2(result_hi, T1Q, b, a, T2D);  // A1*B1
+  eor(tmp1, T16B, tmp1, b);           // (B1+B0)
+  pmull(result_lo,  T1Q, b, a, T1D);  // A0*B0
+  pmull(tmp2, T1Q, tmp1, a1_xor_a0, T1D); // (A1+A0)(B1+B0)
+
+  ext(tmp1, T16B, result_lo, result_hi, 0x08);
+  eor(tmp3, T16B, result_hi, result_lo); // A1*B1+A0*B0
+  eor(tmp2, T16B, tmp2, tmp1);
+  eor(tmp2, T16B, tmp2, tmp3);
+
+  // Register pair <result_hi:result_lo> holds the result of carry-less multiplication
+  ins(result_hi, D, tmp2, 0, 1);
+  ins(result_lo, D, tmp2, 1, 0);
+}
+
+void MacroAssembler::ghash_reduce(FloatRegister result, FloatRegister lo, FloatRegister hi,
+                  FloatRegister p, FloatRegister vzr, FloatRegister t1) {
+  const FloatRegister t0 = result;
+
+  // The GCM field polynomial f is z^128 + p(z), where p =
+  // z^7+z^2+z+1.
+  //
+  //    z^128 === -p(z)  (mod (z^128 + p(z)))
+  //
+  // so, given that the product we're reducing is
+  //    a == lo + hi * z^128
+  // substituting,
+  //      === lo - hi * p(z)  (mod (z^128 + p(z)))
+  //
+  // we reduce by multiplying hi by p(z) and subtracting the result
+  // from (i.e. XORing it with) lo.  Because p has no nonzero high
+  // bits we can do this with two 64-bit multiplications, lo*p and
+  // hi*p.
+
+  pmull2(t0, T1Q, hi, p, T2D);
+  ext(t1, T16B, t0, vzr, 8);
+  eor(hi, T16B, hi, t1);
+  ext(t1, T16B, vzr, t0, 8);
+  eor(lo, T16B, lo, t1);
+  pmull(t0, T1Q, hi, p, T1D);
+  eor(result, T16B, lo, t0);
+}
+
+class GHASHMultiplyGenerator: public KernelGenerator {
+  FloatRegister _result_lo, _result_hi, _b,
+    _a, _vzr, _a1_xor_a0, _p,
+    _tmp1, _tmp2, _tmp3;
+
+public:
+  GHASHMultiplyGenerator(Assembler *as, int unrolls,
+                         /* offsetted registers */
+                         FloatRegister result_lo, FloatRegister result_hi,
+                         FloatRegister b,
+                         /* non-offsetted (shared) registers */
+                         FloatRegister a, FloatRegister a1_xor_a0, FloatRegister p, FloatRegister vzr,
+                         /* offseted (temp) registers */
+                         FloatRegister tmp1, FloatRegister tmp2, FloatRegister tmp3)
+    : KernelGenerator(as, unrolls),
+      _result_lo(result_lo), _result_hi(result_hi), _b(b),
+      _a(a), _vzr(vzr), _a1_xor_a0(a1_xor_a0), _p(p),
+      _tmp1(tmp1), _tmp2(tmp2), _tmp3(tmp3) { }
+
+  int register_stride = 7;
+
+  virtual void generate(int index) {
+    // Karatsuba multiplication performs a 128*128 -> 256-bit
+    // multiplication in three 128-bit multiplications and a few
+    // additions.
+    //
+    // (C1:C0) = A1*B1, (D1:D0) = A0*B0, (E1:E0) = (A0+A1)(B0+B1)
+    // (A1:A0)(B1:B0) = C1:(C0+C1+D1+E1):(D1+C0+D0+E0):D0
+    //
+    // Inputs:
+    //
+    // A0 in a.d[0]     (subkey)
+    // A1 in a.d[1]
+    // (A1+A0) in a1_xor_a0.d[0]
+    //
+    // B0 in b.d[0]     (state)
+    // B1 in b.d[1]
+
+    switch (index) {
+      case  0:  ext(_tmp1, T16B, _b, _b, 0x08);  break;
+      case  1:  pmull2(_result_hi, T1Q, _b, _a, T2D);  // A1*B1
+        break;
+      case  2:  eor(_tmp1, T16B, _tmp1, _b);           // (B1+B0)
+        break;
+      case  3:  pmull(_result_lo,  T1Q, _b, _a, T1D);  // A0*B0
+        break;
+      case  4:  pmull(_tmp2, T1Q, _tmp1, _a1_xor_a0, T1D); // (A1+A0)(B1+B0)
+        break;
+
+      case  5:  ext(_tmp1, T16B, _result_lo, _result_hi, 0x08);  break;
+      case  6:  eor(_tmp3, T16B, _result_hi, _result_lo); // A1*B1+A0*B0
+        break;
+      case  7:  eor(_tmp2, T16B, _tmp2, _tmp1);  break;
+      case  8:  eor(_tmp2, T16B, _tmp2, _tmp3);  break;
+
+        // Register pair <_result_hi:_result_lo> holds the _result of carry-less multiplication
+      case  9:  ins(_result_hi, D, _tmp2, 0, 1);  break;
+      case 10:  ins(_result_lo, D, _tmp2, 1, 0);  break;
+      default: ShouldNotReachHere();
+    }
+  }
+
+  virtual KernelGenerator *next() {
+    GHASHMultiplyGenerator *result = new GHASHMultiplyGenerator(*this);
+    result->_result_lo += register_stride;
+    result->_result_hi += register_stride;
+    result->_b += register_stride;
+    result->_tmp1 += register_stride;
+    result->_tmp2 += register_stride;
+    result->_tmp3 += register_stride;
+    return result;
+  }
+
+  virtual int length() { return 11; }
+};
+
+// Reduce the 128-bit product in hi:lo by the GCM field polynomial.
+// The FloatRegister argument called data is optional: if it is a
+// valid register, we interleave LD1 instructions with the
+// reduction. This is to reduce latency next time around the loop.
+class GHASHReduceGenerator: public KernelGenerator {
+  FloatRegister _result, _lo, _hi, _p, _vzr, _data, _t1;
+  int _once;
+public:
+  GHASHReduceGenerator(Assembler *as, int unrolls,
+                       /* offsetted registers */
+                       FloatRegister result, FloatRegister lo, FloatRegister hi,
+                       /* non-offsetted (shared) registers */
+                       FloatRegister p, FloatRegister vzr, FloatRegister data,
+                       /* offseted (temp) registers */
+                       FloatRegister t1)
+    : KernelGenerator(as, unrolls),
+      _result(result), _lo(lo), _hi(hi),
+      _p(p), _vzr(vzr), _data(data), _t1(t1), _once(true) { }
+
+  int register_stride = 7;
+
+  virtual void generate(int index) {
+    const FloatRegister t0 = _result;
+
+    switch (index) {
+      // The GCM field polynomial f is z^128 + p(z), where p =
+      // z^7+z^2+z+1.
+      //
+      //    z^128 === -p(z)  (mod (z^128 + p(z)))
+      //
+      // so, given that the product we're reducing is
+      //    a == lo + hi * z^128
+      // substituting,
+      //      === lo - hi * p(z)  (mod (z^128 + p(z)))
+      //
+      // we reduce by multiplying hi by p(z) and subtracting the _result
+      // from (i.e. XORing it with) lo.  Because p has no nonzero high
+      // bits we can do this with two 64-bit multiplications, lo*p and
+      // hi*p.
+
+      case  0:  pmull2(t0, T1Q, _hi, _p, T2D);  break;
+      case  1:  ext(_t1, T16B, t0, _vzr, 8);  break;
+      case  2:  eor(_hi, T16B, _hi, _t1);  break;
+      case  3:  ext(_t1, T16B, _vzr, t0, 8);  break;
+      case  4:  eor(_lo, T16B, _lo, _t1);  break;
+      case  5:  pmull(t0, T1Q, _hi, _p, T1D);  break;
+      case  6:  eor(_result, T16B, _lo, t0);  break;
+      default: ShouldNotReachHere();
+    }
+
+    // Sprinkle load instructions into the generated instructions
+    if (_data->is_valid() && _once) {
+      assert(length() >= unrolls(), "not enough room for inteleaved loads");
+      if (index < unrolls()) {
+        ld1((_data + index*register_stride), T16B, post(r2, 0x10));
+      }
+    }
+  }
+
+  virtual KernelGenerator *next() {
+    GHASHReduceGenerator *result = new GHASHReduceGenerator(*this);
+    result->_result += register_stride;
+    result->_hi += register_stride;
+    result->_lo += register_stride;
+    result->_t1 += register_stride;
+    result->_once = false;
+    return result;
+  }
+
+ int length() { return 7; }
+};
+
+// Perform a GHASH multiply/reduce on a single FloatRegister.
+void MacroAssembler::ghash_modmul(FloatRegister result,
+                                  FloatRegister result_lo, FloatRegister result_hi, FloatRegister b,
+                                  FloatRegister a, FloatRegister vzr, FloatRegister a1_xor_a0, FloatRegister p,
+                                  FloatRegister t1, FloatRegister t2, FloatRegister t3) {
+  ghash_multiply(result_lo, result_hi, a, b, a1_xor_a0, t1, t2, t3);
+  ghash_reduce(result, result_lo, result_hi, p, vzr, t1);
+}
+
+// Interleaved GHASH processing.
+//
+// Clobbers all vector registers.
+//
+void MacroAssembler::ghash_processBlocks_wide(address field_polynomial, Register state,
+                                              Register subkeyH,
+                                              Register data, Register blocks, int unrolls) {
+  int register_stride = 7;
+
+  // Bafflingly, GCM uses little-endian for the byte order, but
+  // big-endian for the bit order.  For example, the polynomial 1 is
+  // represented as the 16-byte string 80 00 00 00 | 12 bytes of 00.
+  //
+  // So, we must either reverse the bytes in each word and do
+  // everything big-endian or reverse the bits in each byte and do
+  // it little-endian.  On AArch64 it's more idiomatic to reverse
+  // the bits in each byte (we have an instruction, RBIT, to do
+  // that) and keep the data in little-endian bit order throught the
+  // calculation, bit-reversing the inputs and outputs.
+
+  assert(unrolls * register_stride < 32, "out of registers");
+
+  FloatRegister a1_xor_a0 = v28;
+  FloatRegister Hprime = v29;
+  FloatRegister vzr = v30;
+  FloatRegister p = v31;
+  eor(vzr, T16B, vzr, vzr); // zero register
+
+  ldrq(p, field_polynomial);    // The field polynomial
+
+  ldrq(v0, Address(state));
+  ldrq(Hprime, Address(subkeyH));
+
+  rev64(v0, T16B, v0);          // Bit-reverse words in state and subkeyH
+  rbit(v0, T16B, v0);
+  rev64(Hprime, T16B, Hprime);
+  rbit(Hprime, T16B, Hprime);
+
+  // Powers of H -> Hprime
+
+  Label already_calculated, done;
+  {
+    // The first time around we'll have to calculate H**2, H**3, etc.
+    // Look at the largest power of H in the subkeyH array to see if
+    // it's already been calculated.
+    ldp(rscratch1, rscratch2, Address(subkeyH, 16 * (unrolls - 1)));
+    orr(rscratch1, rscratch1, rscratch2);
+    cbnz(rscratch1, already_calculated);
+
+    orr(v6, T16B, Hprime, Hprime);  // Start with H in v6 and Hprime
+    for (int i = 1; i < unrolls; i++) {
+      ext(a1_xor_a0, T16B, Hprime, Hprime, 0x08); // long-swap subkeyH into a1_xor_a0
+      eor(a1_xor_a0, T16B, a1_xor_a0, Hprime);    // xor subkeyH into subkeyL (Karatsuba: (A1+A0))
+      ghash_modmul(/*result*/v6, /*result_lo*/v5, /*result_hi*/v4, /*b*/v6,
+                   Hprime, vzr, a1_xor_a0, p,
+                   /*temps*/v1, v3, v2);
+      rev64(v1, T16B, v6);
+      rbit(v1, T16B, v1);
+      strq(v1, Address(subkeyH, 16 * i));
+    }
+    b(done);
+  }
+  {
+    bind(already_calculated);
+
+    // Load the largest power of H we need into v6.
+    ldrq(v6, Address(subkeyH, 16 * (unrolls - 1)));
+    rev64(v6, T16B, v6);
+    rbit(v6, T16B, v6);
+  }
+  bind(done);
+
+  orr(Hprime, T16B, v6, v6);     // Move H ** unrolls into Hprime
+
+  // Hprime contains (H ** 1, H ** 2, ... H ** unrolls)
+  // v0 contains the initial state. Clear the others.
+  for (int i = 1; i < unrolls; i++) {
+    int ofs = register_stride * i;
+    eor(ofs+v0, T16B, ofs+v0, ofs+v0); // zero each state register
+  }
+
+  ext(a1_xor_a0, T16B, Hprime, Hprime, 0x08); // long-swap subkeyH into a1_xor_a0
+  eor(a1_xor_a0, T16B, a1_xor_a0, Hprime);    // xor subkeyH into subkeyL (Karatsuba: (A1+A0))
+
+  // Load #unrolls blocks of data
+  for (int ofs = 0; ofs < unrolls * register_stride; ofs += register_stride) {
+    ld1(v2+ofs, T16B, post(data, 0x10));
+  }
+
+  // Register assignments, replicated across 4 clones, v0 ... v23
+  //
+  // v0: input / output: current state, result of multiply/reduce
+  // v1: temp
+  // v2: input: one block of data (the ciphertext)
+  //     also used as a temp once the data has been consumed
+  // v3: temp
+  // v4: output: high part of product
+  // v5: output: low part ...
+  // v6: unused
+  //
+  // Not replicated:
+  //
+  // v28: High part of H xor low part of H'
+  // v29: H' (hash subkey)
+  // v30: zero
+  // v31: Reduction polynomial of the Galois field
+
+  // Inner loop.
+  // Do the whole load/add/multiply/reduce over all our data except
+  // the last few rows.
+  {
+    Label L_ghash_loop;
+    bind(L_ghash_loop);
+
+    // Prefetching doesn't help here. In fact, on Neoverse N1 it's worse.
+    // prfm(Address(data, 128), PLDL1KEEP);
+
+    // Xor data into current state
+    for (int ofs = 0; ofs < unrolls * register_stride; ofs += register_stride) {
+      rbit((v2+ofs), T16B, (v2+ofs));
+      eor((v2+ofs), T16B, v0+ofs, (v2+ofs));   // bit-swapped data ^ bit-swapped state
+    }
+
+    // Generate fully-unrolled multiply-reduce in two stages.
+
+    GHASHMultiplyGenerator(this, unrolls,
+                           /*result_lo*/v5, /*result_hi*/v4, /*data*/v2,
+                           Hprime, a1_xor_a0, p, vzr,
+                           /*temps*/v1, v3, /* reuse b*/v2) .unroll();
+
+    // NB: GHASHReduceGenerator also loads the next #unrolls blocks of
+    // data into v0, v0+ofs, the current state.
+    GHASHReduceGenerator (this, unrolls,
+                          /*result*/v0, /*lo*/v5, /*hi*/v4, p, vzr,
+                          /*data*/v2, /*temp*/v3) .unroll();
+
+    sub(blocks, blocks, unrolls);
+    cmp(blocks, (unsigned char)(unrolls * 2));
+    br(GE, L_ghash_loop);
+  }
+
+  // Merge the #unrolls states.  Note that the data for the next
+  // iteration has already been loaded into v4, v4+ofs, etc...
+
+  // First, we multiply/reduce each clone by the appropriate power of H.
+  for (int i = 0; i < unrolls; i++) {
+    int ofs = register_stride * i;
+    ldrq(Hprime, Address(subkeyH, 16 * (unrolls - i - 1)));
+
+    rbit(v2+ofs, T16B, v2+ofs);
+    eor(v2+ofs, T16B, ofs+v0, v2+ofs);   // bit-swapped data ^ bit-swapped state
+
+    rev64(Hprime, T16B, Hprime);
+    rbit(Hprime, T16B, Hprime);
+    ext(a1_xor_a0, T16B, Hprime, Hprime, 0x08); // long-swap subkeyH into a1_xor_a0
+    eor(a1_xor_a0, T16B, a1_xor_a0, Hprime);    // xor subkeyH into subkeyL (Karatsuba: (A1+A0))
+    ghash_modmul(/*result*/v0+ofs, /*result_lo*/v5+ofs, /*result_hi*/v4+ofs, /*b*/v2+ofs,
+                 Hprime, vzr, a1_xor_a0, p,
+                 /*temps*/v1+ofs, v3+ofs, /* reuse b*/v2+ofs);
+  }
+
+  // Then we sum the results.
+  for (int i = 0; i < unrolls - 1; i++) {
+    int ofs = register_stride * i;
+    eor(v0, T16B, v0, v0 + register_stride + ofs);
+  }
+
+  sub(blocks, blocks, (unsigned char)unrolls);
+
+  // And finally bit-reverse the state back to big endian.
+  rev64(v0, T16B, v0);
+  rbit(v0, T16B, v0);
+  st1(v0, T16B, state);
+}

src/hotspot/cpu/aarch64/macroAssembler_aarch64_log.cpp

@@ -1,5 +1,5 @@
 /* Copyright (c) 2018, Cavium. All rights reserved. (By BELLSOFT)
- * Copyright (c) 2016, Intel Corporation.
+ * Copyright (c) 2016, 2021, Intel Corporation. All rights reserved.
  * Intel Math Library (LIBM) Source Code
  *
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.

src/hotspot/cpu/aarch64/matcher_aarch64.hpp

@@ -55,6 +55,9 @@
   // No support for generic vector operands.
   static const bool supports_generic_vector_operands = false;
 
+  // No support for 48 extra htbl entries in aes-gcm intrinsic
+  static const int htbl_entries = 0;
+
   static constexpr bool isSimpleConstant64(jlong value) {
     // Will one (StoreL ConL) be cheaper than two (StoreI ConI)?.
     // Probably always true, even if a temp register is required.
@@ -160,4 +163,7 @@
       return true;
   }
 
+  // Implements a variant of EncodeISOArrayNode that encode ASCII only
+  static const bool supports_encode_ascii_array = false;
+
 #endif // CPU_AARCH64_MATCHER_AARCH64_HPP

src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp

@@ -2560,8 +2560,6 @@ class StubGenerator: public StubCodeGenerator {
     __ align(CodeEntryAlignment);
     StubCodeMark mark(this, "StubRoutines", "aescrypt_encryptBlock");
 
-    Label L_doLast;
-
     const Register from        = c_rarg0;  // source array address
     const Register to          = c_rarg1;  // destination array address
     const Register key         = c_rarg2;  // key array address
@@ -2572,75 +2570,8 @@ class StubGenerator: public StubCodeGenerator {
 
     __ ldrw(keylen, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT)));
 
-    __ ld1(v0, __ T16B, from); // get 16 bytes of input
-
-    __ ld1(v1, v2, v3, v4, __ T16B, __ post(key, 64));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-    __ rev32(v3, __ T16B, v3);
-    __ rev32(v4, __ T16B, v4);
-    __ aese(v0, v1);
-    __ aesmc(v0, v0);
-    __ aese(v0, v2);
-    __ aesmc(v0, v0);
-    __ aese(v0, v3);
-    __ aesmc(v0, v0);
-    __ aese(v0, v4);
-    __ aesmc(v0, v0);
-
-    __ ld1(v1, v2, v3, v4, __ T16B, __ post(key, 64));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-    __ rev32(v3, __ T16B, v3);
-    __ rev32(v4, __ T16B, v4);
-    __ aese(v0, v1);
-    __ aesmc(v0, v0);
-    __ aese(v0, v2);
-    __ aesmc(v0, v0);
-    __ aese(v0, v3);
-    __ aesmc(v0, v0);
-    __ aese(v0, v4);
-    __ aesmc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ cmpw(keylen, 44);
-    __ br(Assembler::EQ, L_doLast);
-
-    __ aese(v0, v1);
-    __ aesmc(v0, v0);
-    __ aese(v0, v2);
-    __ aesmc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ cmpw(keylen, 52);
-    __ br(Assembler::EQ, L_doLast);
-
-    __ aese(v0, v1);
-    __ aesmc(v0, v0);
-    __ aese(v0, v2);
-    __ aesmc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ BIND(L_doLast);
-
-    __ aese(v0, v1);
-    __ aesmc(v0, v0);
-    __ aese(v0, v2);
-
-    __ ld1(v1, __ T16B, key);
-    __ rev32(v1, __ T16B, v1);
-    __ eor(v0, __ T16B, v0, v1);
-
-    __ st1(v0, __ T16B, to);
+    __ aesenc_loadkeys(key, keylen);
+    __ aesecb_encrypt(from, to, keylen);
 
     __ mov(r0, 0);
 
@@ -2673,76 +2604,7 @@ class StubGenerator: public StubCodeGenerator {
 
     __ ldrw(keylen, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT)));
 
-    __ ld1(v0, __ T16B, from); // get 16 bytes of input
-
-    __ ld1(v5, __ T16B, __ post(key, 16));
-    __ rev32(v5, __ T16B, v5);
-
-    __ ld1(v1, v2, v3, v4, __ T16B, __ post(key, 64));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-    __ rev32(v3, __ T16B, v3);
-    __ rev32(v4, __ T16B, v4);
-    __ aesd(v0, v1);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v2);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v3);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v4);
-    __ aesimc(v0, v0);
-
-    __ ld1(v1, v2, v3, v4, __ T16B, __ post(key, 64));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-    __ rev32(v3, __ T16B, v3);
-    __ rev32(v4, __ T16B, v4);
-    __ aesd(v0, v1);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v2);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v3);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v4);
-    __ aesimc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ cmpw(keylen, 44);
-    __ br(Assembler::EQ, L_doLast);
-
-    __ aesd(v0, v1);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v2);
-    __ aesimc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ cmpw(keylen, 52);
-    __ br(Assembler::EQ, L_doLast);
-
-    __ aesd(v0, v1);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v2);
-    __ aesimc(v0, v0);
-
-    __ ld1(v1, v2, __ T16B, __ post(key, 32));
-    __ rev32(v1, __ T16B, v1);
-    __ rev32(v2, __ T16B, v2);
-
-    __ BIND(L_doLast);
-
-    __ aesd(v0, v1);
-    __ aesimc(v0, v0);
-    __ aesd(v0, v2);
-
-    __ eor(v0, __ T16B, v0, v5);
-
-    __ st1(v0, __ T16B, to);
+    __ aesecb_decrypt(from, to, key, keylen);
 
     __ mov(r0, 0);
 
@@ -2964,6 +2826,390 @@ class StubGenerator: public StubCodeGenerator {
     return start;
   }
 
+  // CTR AES crypt.
+  // Arguments:
+  //
+  // Inputs:
+  //   c_rarg0   - source byte array address
+  //   c_rarg1   - destination byte array address
+  //   c_rarg2   - K (key) in little endian int array
+  //   c_rarg3   - counter vector byte array address
+  //   c_rarg4   - input length
+  //   c_rarg5   - saved encryptedCounter start
+  //   c_rarg6   - saved used length
+  //
+  // Output:
+  //   r0       - input length
+  //
+  address generate_counterMode_AESCrypt() {
+    const Register in = c_rarg0;
+    const Register out = c_rarg1;
+    const Register key = c_rarg2;
+    const Register counter = c_rarg3;
+    const Register saved_len = c_rarg4, len = r10;
+    const Register saved_encrypted_ctr = c_rarg5;
+    const Register used_ptr = c_rarg6, used = r12;
+
+    const Register offset = r7;
+    const Register keylen = r11;
+
+    const unsigned char block_size = 16;
+    const int bulk_width = 4;
+    // NB: bulk_width can be 4 or 8. 8 gives slightly faster
+    // performance with larger data sizes, but it also means that the
+    // fast path isn't used until you have at least 8 blocks, and up
+    // to 127 bytes of data will be executed on the slow path. For
+    // that reason, and also so as not to blow away too much icache, 4
+    // blocks seems like a sensible compromise.
+
+    // Algorithm:
+    //
+    //    if (len == 0) {
+    //        goto DONE;
+    //    }
+    //    int result = len;
+    //    do {
+    //        if (used >= blockSize) {
+    //            if (len >= bulk_width * blockSize) {
+    //                CTR_large_block();
+    //                if (len == 0)
+    //                    goto DONE;
+    //            }
+    //            for (;;) {
+    //                16ByteVector v0 = counter;
+    //                embeddedCipher.encryptBlock(v0, 0, encryptedCounter, 0);
+    //                used = 0;
+    //                if (len < blockSize)
+    //                    break;    /* goto NEXT */
+    //                16ByteVector v1 = load16Bytes(in, offset);
+    //                v1 = v1 ^ encryptedCounter;
+    //                store16Bytes(out, offset);
+    //                used = blockSize;
+    //                offset += blockSize;
+    //                len -= blockSize;
+    //                if (len == 0)
+    //                    goto DONE;
+    //            }
+    //        }
+    //      NEXT:
+    //        out[outOff++] = (byte)(in[inOff++] ^ encryptedCounter[used++]);
+    //        len--;
+    //    } while (len != 0);
+    //  DONE:
+    //    return result;
+    //
+    // CTR_large_block()
+    //    Wide bulk encryption of whole blocks.
+
+    __ align(CodeEntryAlignment);
+    StubCodeMark mark(this, "StubRoutines", "counterMode_AESCrypt");
+    const address start = __ pc();
+    __ enter();
+
+    Label DONE, CTR_large_block, large_block_return;
+    __ ldrw(used, Address(used_ptr));
+    __ cbzw(saved_len, DONE);
+
+    __ mov(len, saved_len);
+    __ mov(offset, 0);
+
+    // Compute #rounds for AES based on the length of the key array
+    __ ldrw(keylen, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT)));
+
+    __ aesenc_loadkeys(key, keylen);
+
+    {
+      Label L_CTR_loop, NEXT;
+
+      __ bind(L_CTR_loop);
+
+      __ cmp(used, block_size);
+      __ br(__ LO, NEXT);
+
+      // Maybe we have a lot of data
+      __ subsw(rscratch1, len, bulk_width * block_size);
+      __ br(__ HS, CTR_large_block);
+      __ BIND(large_block_return);
+      __ cbzw(len, DONE);
+
+      // Setup the counter
+      __ movi(v4, __ T4S, 0);
+      __ movi(v5, __ T4S, 1);
+      __ ins(v4, __ S, v5, 3, 3); // v4 contains { 0, 0, 0, 1 }
+
+      __ ld1(v0, __ T16B, counter); // Load the counter into v0
+      __ rev32(v16, __ T16B, v0);
+      __ addv(v16, __ T4S, v16, v4);
+      __ rev32(v16, __ T16B, v16);
+      __ st1(v16, __ T16B, counter); // Save the incremented counter back
+
+      {
+        // We have fewer than bulk_width blocks of data left. Encrypt
+        // them one by one until there is less than a full block
+        // remaining, being careful to save both the encrypted counter
+        // and the counter.
+
+        Label inner_loop;
+        __ bind(inner_loop);
+        // Counter to encrypt is in v0
+        __ aesecb_encrypt(noreg, noreg, keylen);
+        __ st1(v0, __ T16B, saved_encrypted_ctr);
+
+        // Do we have a remaining full block?
+
+        __ mov(used, 0);
+        __ cmp(len, block_size);
+        __ br(__ LO, NEXT);
+
+        // Yes, we have a full block
+        __ ldrq(v1, Address(in, offset));
+        __ eor(v1, __ T16B, v1, v0);
+        __ strq(v1, Address(out, offset));
+        __ mov(used, block_size);
+        __ add(offset, offset, block_size);
+
+        __ subw(len, len, block_size);
+        __ cbzw(len, DONE);
+
+        // Increment the counter, store it back
+        __ orr(v0, __ T16B, v16, v16);
+        __ rev32(v16, __ T16B, v16);
+        __ addv(v16, __ T4S, v16, v4);
+        __ rev32(v16, __ T16B, v16);
+        __ st1(v16, __ T16B, counter); // Save the incremented counter back
+
+        __ b(inner_loop);
+      }
+
+      __ BIND(NEXT);
+
+      // Encrypt a single byte, and loop.
+      // We expect this to be a rare event.
+      __ ldrb(rscratch1, Address(in, offset));
+      __ ldrb(rscratch2, Address(saved_encrypted_ctr, used));
+      __ eor(rscratch1, rscratch1, rscratch2);
+      __ strb(rscratch1, Address(out, offset));
+      __ add(offset, offset, 1);
+      __ add(used, used, 1);
+      __ subw(len, len,1);
+      __ cbnzw(len, L_CTR_loop);
+    }
+
+    __ bind(DONE);
+    __ strw(used, Address(used_ptr));
+    __ mov(r0, saved_len);
+
+    __ leave(); // required for proper stackwalking of RuntimeStub frame
+    __ ret(lr);
+
+    // Bulk encryption
+
+    __ BIND (CTR_large_block);
+    assert(bulk_width == 4 || bulk_width == 8, "must be");
+
+    if (bulk_width == 8) {
+      __ sub(sp, sp, 4 * 16);
+      __ st1(v12, v13, v14, v15, __ T16B, Address(sp));
+    }
+    __ sub(sp, sp, 4 * 16);
+    __ st1(v8, v9, v10, v11, __ T16B, Address(sp));
+    RegSet saved_regs = (RegSet::of(in, out, offset)
+                         + RegSet::of(saved_encrypted_ctr, used_ptr, len));
+    __ push(saved_regs, sp);
+    __ andr(len, len, -16 * bulk_width);  // 8/4 encryptions, 16 bytes per encryption
+    __ add(in, in, offset);
+    __ add(out, out, offset);
+
+    // Keys should already be loaded into the correct registers
+
+    __ ld1(v0, __ T16B, counter); // v0 contains the first counter
+    __ rev32(v16, __ T16B, v0); // v16 contains byte-reversed counter
+
+    // AES/CTR loop
+    {
+      Label L_CTR_loop;
+      __ BIND(L_CTR_loop);
+
+      // Setup the counters
+      __ movi(v8, __ T4S, 0);
+      __ movi(v9, __ T4S, 1);
+      __ ins(v8, __ S, v9, 3, 3); // v8 contains { 0, 0, 0, 1 }
+
+      for (FloatRegister f = v0; f < v0 + bulk_width; f++) {
+        __ rev32(f, __ T16B, v16);
+        __ addv(v16, __ T4S, v16, v8);
+      }
+
+      __ ld1(v8, v9, v10, v11, __ T16B, __ post(in, 4 * 16));
+
+      // Encrypt the counters
+      __ aesecb_encrypt(noreg, noreg, keylen, v0, bulk_width);
+
+      if (bulk_width == 8) {
+        __ ld1(v12, v13, v14, v15, __ T16B, __ post(in, 4 * 16));
+      }
+
+      // XOR the encrypted counters with the inputs
+      for (int i = 0; i < bulk_width; i++) {
+        __ eor(v0 + i, __ T16B, v0 + i, v8 + i);
+      }
+
+      // Write the encrypted data
+      __ st1(v0, v1, v2, v3, __ T16B, __ post(out, 4 * 16));
+      if (bulk_width == 8) {
+        __ st1(v4, v5, v6, v7, __ T16B, __ post(out, 4 * 16));
+      }
+
+      __ subw(len, len, 16 * bulk_width);
+      __ cbnzw(len, L_CTR_loop);
+    }
+
+    // Save the counter back where it goes
+    __ rev32(v16, __ T16B, v16);
+    __ st1(v16, __ T16B, counter);
+
+    __ pop(saved_regs, sp);
+
+    __ ld1(v8, v9, v10, v11, __ T16B, __ post(sp, 4 * 16));
+    if (bulk_width == 8) {
+      __ ld1(v12, v13, v14, v15, __ T16B, __ post(sp, 4 * 16));
+    }
+
+    __ andr(rscratch1, len, -16 * bulk_width);
+    __ sub(len, len, rscratch1);
+    __ add(offset, offset, rscratch1);
+    __ mov(used, 16);
+    __ strw(used, Address(used_ptr));
+    __ b(large_block_return);
+
+    return start;
+  }
+
+  // Vector AES Galois Counter Mode implementation. Parameters:
+  //
+  // in = c_rarg0
+  // len = c_rarg1
+  // ct = c_rarg2 - ciphertext that ghash will read (in for encrypt, out for decrypt)
+  // out = c_rarg3
+  // key = c_rarg4
+  // state = c_rarg5 - GHASH.state
+  // subkeyHtbl = c_rarg6 - powers of H
+  // subkeyHtbl_48_entries = c_rarg7 (not used)
+  // counter = [sp, #0] pointer to 16 bytes of CTR
+  // return - number of processed bytes
+  address generate_galoisCounterMode_AESCrypt() {
+    address ghash_polynomial = __ pc();
+    __ emit_int64(0x87);  // The low-order bits of the field
+                          // polynomial (i.e. p = z^7+z^2+z+1)
+                          // repeated in the low and high parts of a
+                          // 128-bit vector
+    __ emit_int64(0x87);
+
+    __ align(CodeEntryAlignment);
+     StubCodeMark mark(this, "StubRoutines", "galoisCounterMode_AESCrypt");
+    address start = __ pc();
+    __ enter();
+
+    const Register in = c_rarg0;
+    const Register len = c_rarg1;
+    const Register ct = c_rarg2;
+    const Register out = c_rarg3;
+    // and updated with the incremented counter in the end
+
+    const Register key = c_rarg4;
+    const Register state = c_rarg5;
+
+    const Register subkeyHtbl = c_rarg6;
+
+    // Pointer to CTR is passed on the stack before the (fp, lr) pair.
+    const Address counter_mem(sp, 2 * wordSize);
+    const Register counter = c_rarg7;
+    __ ldr(counter, counter_mem);
+
+    const Register keylen = r10;
+    // Save state before entering routine
+    __ sub(sp, sp, 4 * 16);
+    __ st1(v12, v13, v14, v15, __ T16B, Address(sp));
+    __ sub(sp, sp, 4 * 16);
+    __ st1(v8, v9, v10, v11, __ T16B, Address(sp));
+
+    // __ andr(len, len, -512);
+    __ andr(len, len, -16 * 8);  // 8 encryptions, 16 bytes per encryption
+    __ str(len, __ pre(sp, -2 * wordSize));
+
+    Label DONE;
+    __ cbz(len, DONE);
+
+    // Compute #rounds for AES based on the length of the key array
+    __ ldrw(keylen, Address(key, arrayOopDesc::length_offset_in_bytes() - arrayOopDesc::base_offset_in_bytes(T_INT)));
+
+    __ aesenc_loadkeys(key, keylen);
+    __ ld1(v0, __ T16B, counter); // v0 contains the first counter
+    __ rev32(v16, __ T16B, v0); // v16 contains byte-reversed counter
+
+    // AES/CTR loop
+    {
+      Label L_CTR_loop;
+      __ BIND(L_CTR_loop);
+
+      // Setup the counters
+      __ movi(v8, __ T4S, 0);
+      __ movi(v9, __ T4S, 1);
+      __ ins(v8, __ S, v9, 3, 3); // v8 contains { 0, 0, 0, 1 }
+      for (FloatRegister f = v0; f < v8; f++) {
+        __ rev32(f, __ T16B, v16);
+        __ addv(v16, __ T4S, v16, v8);
+      }
+
+      __ ld1(v8, v9, v10, v11, __ T16B, __ post(in, 4 * 16));
+
+      // Encrypt the counters
+      __ aesecb_encrypt(noreg, noreg, keylen, v0, /*unrolls*/8);
+
+      __ ld1(v12, v13, v14, v15, __ T16B, __ post(in, 4 * 16));
+
+      // XOR the encrypted counters with the inputs
+      for (int i = 0; i < 8; i++) {
+        __ eor(v0 + i, __ T16B, v0 + i, v8 + i);
+      }
+      __ st1(v0, v1, v2, v3, __ T16B, __ post(out, 4 * 16));
+      __ st1(v4, v5, v6, v7, __ T16B, __ post(out, 4 * 16));
+
+      __ subw(len, len, 16 * 8);
+      __ cbnzw(len, L_CTR_loop);
+    }
+
+    __ rev32(v16, __ T16B, v16);
+    __ st1(v16, __ T16B, counter);
+
+    __ ldr(len, Address(sp));
+    __ lsr(len, len, exact_log2(16));  // We want the count of blocks
+
+    // GHASH/CTR loop
+    __ ghash_processBlocks_wide(ghash_polynomial, state, subkeyHtbl, ct,
+                                len, /*unrolls*/4);
+
+#ifdef ASSERT
+    { Label L;
+      __ cmp(len, (unsigned char)0);
+      __ br(Assembler::EQ, L);
+      __ stop("stubGenerator: abort");
+      __ bind(L);
+  }
+#endif
+
+  __ bind(DONE);
+    // Return the number of bytes processed
+    __ ldr(r0, __ post(sp, 2 * wordSize));
+
+    __ ld1(v8, v9, v10, v11, __ T16B, __ post(sp, 4 * 16));
+    __ ld1(v12, v13, v14, v15, __ T16B, __ post(sp, 4 * 16));
+
+    __ leave(); // required for proper stackwalking of RuntimeStub frame
+    __ ret(lr);
+     return start;
+  }
+
   // Arguments:
   //
   // Inputs:
@@ -4227,69 +4473,6 @@ class StubGenerator: public StubCodeGenerator {
     return start;
   }
 
-  void ghash_multiply(FloatRegister result_lo, FloatRegister result_hi,
-                      FloatRegister a, FloatRegister b, FloatRegister a1_xor_a0,
-                      FloatRegister tmp1, FloatRegister tmp2, FloatRegister tmp3, FloatRegister tmp4) {
-    // Karatsuba multiplication performs a 128*128 -> 256-bit
-    // multiplication in three 128-bit multiplications and a few
-    // additions.
-    //
-    // (C1:C0) = A1*B1, (D1:D0) = A0*B0, (E1:E0) = (A0+A1)(B0+B1)
-    // (A1:A0)(B1:B0) = C1:(C0+C1+D1+E1):(D1+C0+D0+E0):D0
-    //
-    // Inputs:
-    //
-    // A0 in a.d[0]     (subkey)
-    // A1 in a.d[1]
-    // (A1+A0) in a1_xor_a0.d[0]
-    //
-    // B0 in b.d[0]     (state)
-    // B1 in b.d[1]
-
-    __ ext(tmp1, __ T16B, b, b, 0x08);
-    __ pmull2(result_hi, __ T1Q, b, a, __ T2D);  // A1*B1
-    __ eor(tmp1, __ T16B, tmp1, b);            // (B1+B0)
-    __ pmull(result_lo,  __ T1Q, b, a, __ T1D);  // A0*B0
-    __ pmull(tmp2, __ T1Q, tmp1, a1_xor_a0, __ T1D); // (A1+A0)(B1+B0)
-
-    __ ext(tmp4, __ T16B, result_lo, result_hi, 0x08);
-    __ eor(tmp3, __ T16B, result_hi, result_lo); // A1*B1+A0*B0
-    __ eor(tmp2, __ T16B, tmp2, tmp4);
-    __ eor(tmp2, __ T16B, tmp2, tmp3);
-
-    // Register pair <result_hi:result_lo> holds the result of carry-less multiplication
-    __ ins(result_hi, __ D, tmp2, 0, 1);
-    __ ins(result_lo, __ D, tmp2, 1, 0);
-  }
-
-  void ghash_reduce(FloatRegister result, FloatRegister lo, FloatRegister hi,
-                    FloatRegister p, FloatRegister z, FloatRegister t1) {
-    const FloatRegister t0 = result;
-
-    // The GCM field polynomial f is z^128 + p(z), where p =
-    // z^7+z^2+z+1.
-    //
-    //    z^128 === -p(z)  (mod (z^128 + p(z)))
-    //
-    // so, given that the product we're reducing is
-    //    a == lo + hi * z^128
-    // substituting,
-    //      === lo - hi * p(z)  (mod (z^128 + p(z)))
-    //
-    // we reduce by multiplying hi by p(z) and subtracting the result
-    // from (i.e. XORing it with) lo.  Because p has no nonzero high
-    // bits we can do this with two 64-bit multiplications, lo*p and
-    // hi*p.
-
-    __ pmull2(t0, __ T1Q, hi, p, __ T2D);
-    __ ext(t1, __ T16B, t0, z, 8);
-    __ eor(hi, __ T16B, hi, t1);
-    __ ext(t1, __ T16B, z, t0, 8);
-    __ eor(lo, __ T16B, lo, t1);
-    __ pmull(t0, __ T1Q, hi, p, __ T1D);
-    __ eor(result, __ T16B, lo, t0);
-  }
-
   address generate_has_negatives(address &has_negatives_long) {
     const u1 large_loop_size = 64;
     const uint64_t UPPER_BIT_MASK=0x8080808080808080;
@@ -4656,18 +4839,6 @@ class StubGenerator: public StubCodeGenerator {
     return entry;
   }
 
-  // code for comparing 16 bytes of strings with same encoding
-  void compare_string_16_bytes_same(Label &DIFF1, Label &DIFF2) {
-    Register result = r0, str1 = r1, cnt1 = r2, str2 = r3, tmp1 = r10, tmp2 = r11;
-    __ ldr(rscratch1, Address(__ post(str1, 8)));
-    __ eor(rscratch2, tmp1, tmp2);
-    __ ldr(cnt1, Address(__ post(str2, 8)));
-    __ cbnz(rscratch2, DIFF1);
-    __ ldr(tmp1, Address(__ post(str1, 8)));
-    __ eor(rscratch2, rscratch1, cnt1);
-    __ ldr(tmp2, Address(__ post(str2, 8)));
-    __ cbnz(rscratch2, DIFF2);
-  }
 
   // code for comparing 16 characters of strings with Latin1 and Utf16 encoding
   void compare_string_16_x_LU(Register tmpL, Register tmpU, Label &DIFF1,
@@ -4874,15 +5045,18 @@ class StubGenerator: public StubCodeGenerator {
         : "compare_long_string_same_encoding UU");
     address entry = __ pc();
     Register result = r0, str1 = r1, cnt1 = r2, str2 = r3, cnt2 = r4,
-        tmp1 = r10, tmp2 = r11;
-    Label SMALL_LOOP, LARGE_LOOP_PREFETCH, CHECK_LAST, DIFF2, TAIL,
-        LENGTH_DIFF, DIFF, LAST_CHECK_AND_LENGTH_DIFF,
-        DIFF_LAST_POSITION, DIFF_LAST_POSITION2;
+        tmp1 = r10, tmp2 = r11, tmp1h = rscratch1, tmp2h = rscratch2;
+
+    Label LARGE_LOOP_PREFETCH, LOOP_COMPARE16, DIFF, LESS16, LESS8, CAL_DIFFERENCE, LENGTH_DIFF;
+
     // exit from large loop when less than 64 bytes left to read or we're about
     // to prefetch memory behind array border
     int largeLoopExitCondition = MAX2(64, SoftwarePrefetchHintDistance)/(isLL ? 1 : 2);
-    // cnt1/cnt2 contains amount of characters to compare. cnt1 can be re-used
-    // update cnt2 counter with already loaded 8 bytes
+
+    // before jumping to stub, pre-load 8 bytes already, so do comparison directly
+    __ eor(rscratch2, tmp1, tmp2);
+    __ cbnz(rscratch2, CAL_DIFFERENCE);
+
     __ sub(cnt2, cnt2, wordSize/(isLL ? 1 : 2));
     // update pointers, because of previous read
     __ add(str1, str1, wordSize);
@@ -4891,80 +5065,88 @@ class StubGenerator: public StubCodeGenerator {
       __ bind(LARGE_LOOP_PREFETCH);
         __ prfm(Address(str1, SoftwarePrefetchHintDistance));
         __ prfm(Address(str2, SoftwarePrefetchHintDistance));
-        compare_string_16_bytes_same(DIFF, DIFF2);
-        compare_string_16_bytes_same(DIFF, DIFF2);
+
+        __ align(OptoLoopAlignment);
+        for (int i = 0; i < 4; i++) {
+          __ ldp(tmp1, tmp1h, Address(str1, i * 16));
+          __ ldp(tmp2, tmp2h, Address(str2, i * 16));
+          __ cmp(tmp1, tmp2);
+          __ ccmp(tmp1h, tmp2h, 0, Assembler::EQ);
+          __ br(Assembler::NE, DIFF);
+        }
         __ sub(cnt2, cnt2, isLL ? 64 : 32);
-        compare_string_16_bytes_same(DIFF, DIFF2);
+        __ add(str1, str1, 64);
+        __ add(str2, str2, 64);
         __ subs(rscratch2, cnt2, largeLoopExitCondition);
-        compare_string_16_bytes_same(DIFF, DIFF2);
-        __ br(__ GT, LARGE_LOOP_PREFETCH);
-        __ cbz(cnt2, LAST_CHECK_AND_LENGTH_DIFF); // no more chars left?
+        __ br(Assembler::GE, LARGE_LOOP_PREFETCH);
+        __ cbz(cnt2, LENGTH_DIFF); // no more chars left?
     }
-    // less than 16 bytes left?
-    __ subs(cnt2, cnt2, isLL ? 16 : 8);
-    __ br(__ LT, TAIL);
+
+    __ subs(rscratch1, cnt2, isLL ? 16 : 8);
+    __ br(Assembler::LE, LESS16);
     __ align(OptoLoopAlignment);
-    __ bind(SMALL_LOOP);
-      compare_string_16_bytes_same(DIFF, DIFF2);
-      __ subs(cnt2, cnt2, isLL ? 16 : 8);
-      __ br(__ GE, SMALL_LOOP);
-    __ bind(TAIL);
-      __ adds(cnt2, cnt2, isLL ? 16 : 8);
-      __ br(__ EQ, LAST_CHECK_AND_LENGTH_DIFF);
+    __ bind(LOOP_COMPARE16);
+      __ ldp(tmp1, tmp1h, Address(__ post(str1, 16)));
+      __ ldp(tmp2, tmp2h, Address(__ post(str2, 16)));
+      __ cmp(tmp1, tmp2);
+      __ ccmp(tmp1h, tmp2h, 0, Assembler::EQ);
+      __ br(Assembler::NE, DIFF);
+      __ sub(cnt2, cnt2, isLL ? 16 : 8);
+      __ subs(rscratch2, cnt2, isLL ? 16 : 8);
+      __ br(Assembler::LT, LESS16);
+
+      __ ldp(tmp1, tmp1h, Address(__ post(str1, 16)));
+      __ ldp(tmp2, tmp2h, Address(__ post(str2, 16)));
+      __ cmp(tmp1, tmp2);
+      __ ccmp(tmp1h, tmp2h, 0, Assembler::EQ);
+      __ br(Assembler::NE, DIFF);
+      __ sub(cnt2, cnt2, isLL ? 16 : 8);
+      __ subs(rscratch2, cnt2, isLL ? 16 : 8);
+      __ br(Assembler::GE, LOOP_COMPARE16);
+      __ cbz(cnt2, LENGTH_DIFF);
+
+    __ bind(LESS16);
+      // each 8 compare
       __ subs(cnt2, cnt2, isLL ? 8 : 4);
-      __ br(__ LE, CHECK_LAST);
-      __ eor(rscratch2, tmp1, tmp2);
-      __ cbnz(rscratch2, DIFF);
+      __ br(Assembler::LE, LESS8);
       __ ldr(tmp1, Address(__ post(str1, 8)));
       __ ldr(tmp2, Address(__ post(str2, 8)));
-      __ sub(cnt2, cnt2, isLL ? 8 : 4);
-    __ bind(CHECK_LAST);
-      if (!isLL) {
-        __ add(cnt2, cnt2, cnt2); // now in bytes
-      }
       __ eor(rscratch2, tmp1, tmp2);
-      __ cbnz(rscratch2, DIFF);
-      __ ldr(rscratch1, Address(str1, cnt2));
-      __ ldr(cnt1, Address(str2, cnt2));
-      __ eor(rscratch2, rscratch1, cnt1);
-      __ cbz(rscratch2, LENGTH_DIFF);
-      // Find the first different characters in the longwords and
-      // compute their difference.
-    __ bind(DIFF2);
-      __ rev(rscratch2, rscratch2);
-      __ clz(rscratch2, rscratch2);
-      __ andr(rscratch2, rscratch2, isLL ? -8 : -16);
-      __ lsrv(rscratch1, rscratch1, rscratch2);
-      if (isLL) {
-        __ lsrv(cnt1, cnt1, rscratch2);
-        __ uxtbw(rscratch1, rscratch1);
-        __ uxtbw(cnt1, cnt1);
-      } else {
-        __ lsrv(cnt1, cnt1, rscratch2);
-        __ uxthw(rscratch1, rscratch1);
-        __ uxthw(cnt1, cnt1);
+      __ cbnz(rscratch2, CAL_DIFFERENCE);
+      __ sub(cnt2, cnt2, isLL ? 8 : 4);
+
+    __ bind(LESS8); // directly load last 8 bytes
+      if (!isLL) {
+        __ add(cnt2, cnt2, cnt2);
       }
-      __ subw(result, rscratch1, cnt1);
-      __ b(LENGTH_DIFF);
+      __ ldr(tmp1, Address(str1, cnt2));
+      __ ldr(tmp2, Address(str2, cnt2));
+      __ eor(rscratch2, tmp1, tmp2);
+      __ cbz(rscratch2, LENGTH_DIFF);
+      __ b(CAL_DIFFERENCE);
+
     __ bind(DIFF);
+      __ cmp(tmp1, tmp2);
+      __ csel(tmp1, tmp1, tmp1h, Assembler::NE);
+      __ csel(tmp2, tmp2, tmp2h, Assembler::NE);
+      // reuse rscratch2 register for the result of eor instruction
+      __ eor(rscratch2, tmp1, tmp2);
+
+    __ bind(CAL_DIFFERENCE);
       __ rev(rscratch2, rscratch2);
       __ clz(rscratch2, rscratch2);
       __ andr(rscratch2, rscratch2, isLL ? -8 : -16);
       __ lsrv(tmp1, tmp1, rscratch2);
+      __ lsrv(tmp2, tmp2, rscratch2);
       if (isLL) {
-        __ lsrv(tmp2, tmp2, rscratch2);
         __ uxtbw(tmp1, tmp1);
         __ uxtbw(tmp2, tmp2);
       } else {
-        __ lsrv(tmp2, tmp2, rscratch2);
         __ uxthw(tmp1, tmp1);
         __ uxthw(tmp2, tmp2);
       }
       __ subw(result, tmp1, tmp2);
-      __ b(LENGTH_DIFF);
-    __ bind(LAST_CHECK_AND_LENGTH_DIFF);
-      __ eor(rscratch2, tmp1, tmp2);
-      __ cbnz(rscratch2, DIFF);
+
     __ bind(LENGTH_DIFF);
       __ ret(lr);
     return entry;
@@ -5387,6 +5569,8 @@ class StubGenerator: public StubCodeGenerator {
     FloatRegister vzr = v30;
     __ eor(vzr, __ T16B, vzr, vzr); // zero register
 
+    __ ldrq(v24, p);    // The field polynomial
+
     __ ldrq(v0, Address(state));
     __ ldrq(v1, Address(subkeyH));
 
@@ -5395,10 +5579,8 @@ class StubGenerator: public StubCodeGenerator {
     __ rev64(v1, __ T16B, v1);
     __ rbit(v1, __ T16B, v1);
 
-    __ ldrq(v26, p);
-
-    __ ext(v16, __ T16B, v1, v1, 0x08); // long-swap subkeyH into v1
-    __ eor(v16, __ T16B, v16, v1);      // xor subkeyH into subkeyL (Karatsuba: (A1+A0))
+    __ ext(v4, __ T16B, v1, v1, 0x08); // long-swap subkeyH into v1
+    __ eor(v4, __ T16B, v4, v1);       // xor subkeyH into subkeyL (Karatsuba: (A1+A0))
 
     {
       Label L_ghash_loop;
@@ -5410,21 +5592,70 @@ class StubGenerator: public StubCodeGenerator {
       __ eor(v2, __ T16B, v0, v2);   // bit-swapped data ^ bit-swapped state
 
       // Multiply state in v2 by subkey in v1
-      ghash_multiply(/*result_lo*/v5, /*result_hi*/v7,
-                     /*a*/v1, /*b*/v2, /*a1_xor_a0*/v16,
-                     /*temps*/v6, v20, v18, v21);
+      __ ghash_multiply(/*result_lo*/v5, /*result_hi*/v7,
+                        /*a*/v1, /*b*/v2, /*a1_xor_a0*/v4,
+                        /*temps*/v6, v3, /*reuse/clobber b*/v2);
       // Reduce v7:v5 by the field polynomial
-      ghash_reduce(v0, v5, v7, v26, vzr, v20);
+      __ ghash_reduce(/*result*/v0, /*lo*/v5, /*hi*/v7, /*p*/v24, vzr, /*temp*/v3);
 
       __ sub(blocks, blocks, 1);
       __ cbnz(blocks, L_ghash_loop);
     }
 
     // The bit-reversed result is at this point in v0
-    __ rev64(v1, __ T16B, v0);
-    __ rbit(v1, __ T16B, v1);
+    __ rev64(v0, __ T16B, v0);
+    __ rbit(v0, __ T16B, v0);
+
+    __ st1(v0, __ T16B, state);
+    __ ret(lr);
+
+    return start;
+  }
+
+  address generate_ghash_processBlocks_wide() {
+    address small = generate_ghash_processBlocks();
+
+    StubCodeMark mark(this, "StubRoutines", "ghash_processBlocks_wide");
+    __ align(wordSize * 2);
+    address p = __ pc();
+    __ emit_int64(0x87);  // The low-order bits of the field
+                          // polynomial (i.e. p = z^7+z^2+z+1)
+                          // repeated in the low and high parts of a
+                          // 128-bit vector
+    __ emit_int64(0x87);
+
+    __ align(CodeEntryAlignment);
+    address start = __ pc();
+
+    Register state   = c_rarg0;
+    Register subkeyH = c_rarg1;
+    Register data    = c_rarg2;
+    Register blocks  = c_rarg3;
+
+    const int unroll = 4;
+
+    __ cmp(blocks, (unsigned char)(unroll * 2));
+    __ br(__ LT, small);
+
+    if (unroll > 1) {
+    // Save state before entering routine
+      __ sub(sp, sp, 4 * 16);
+      __ st1(v12, v13, v14, v15, __ T16B, Address(sp));
+      __ sub(sp, sp, 4 * 16);
+      __ st1(v8, v9, v10, v11, __ T16B, Address(sp));
+    }
+
+    __ ghash_processBlocks_wide(p, state, subkeyH, data, blocks, unroll);
+
+    if (unroll > 1) {
+      // And restore state
+      __ ld1(v8, v9, v10, v11, __ T16B, __ post(sp, 4 * 16));
+      __ ld1(v12, v13, v14, v15, __ T16B, __ post(sp, 4 * 16));
+    }
+
+    __ cmp(blocks, (unsigned char)0);
+    __ br(__ GT, small);
 
-    __ st1(v1, __ T16B, state);
     __ ret(lr);
 
     return start;
@@ -5978,10 +6209,16 @@ class StubGenerator: public StubCodeGenerator {
     __ ret(lr);
   }
 
-  void gen_ldaddal_entry(Assembler::operand_size size) {
+  void gen_ldadd_entry(Assembler::operand_size size, atomic_memory_order order) {
     Register prev = r2, addr = c_rarg0, incr = c_rarg1;
-    __ ldaddal(size, incr, prev, addr);
-    __ membar(Assembler::StoreStore|Assembler::StoreLoad);
+    // If not relaxed, then default to conservative.  Relaxed is the only
+    // case we use enough to be worth specializing.
+    if (order == memory_order_relaxed) {
+      __ ldadd(size, incr, prev, addr);
+    } else {
+      __ ldaddal(size, incr, prev, addr);
+      __ membar(Assembler::StoreStore|Assembler::StoreLoad);
+    }
     if (size == Assembler::xword) {
       __ mov(r0, prev);
     } else {
@@ -6011,12 +6248,21 @@ class StubGenerator: public StubCodeGenerator {
     StubCodeMark mark(this, "StubRoutines", "atomic entry points");
     address first_entry = __ pc();
 
-    // All memory_order_conservative
+    // ADD, memory_order_conservative
     AtomicStubMark mark_fetch_add_4(_masm, &aarch64_atomic_fetch_add_4_impl);
-    gen_ldaddal_entry(Assembler::word);
+    gen_ldadd_entry(Assembler::word, memory_order_conservative);
     AtomicStubMark mark_fetch_add_8(_masm, &aarch64_atomic_fetch_add_8_impl);
-    gen_ldaddal_entry(Assembler::xword);
+    gen_ldadd_entry(Assembler::xword, memory_order_conservative);
 
+    // ADD, memory_order_relaxed
+    AtomicStubMark mark_fetch_add_4_relaxed
+      (_masm, &aarch64_atomic_fetch_add_4_relaxed_impl);
+    gen_ldadd_entry(MacroAssembler::word, memory_order_relaxed);
+    AtomicStubMark mark_fetch_add_8_relaxed
+      (_masm, &aarch64_atomic_fetch_add_8_relaxed_impl);
+    gen_ldadd_entry(MacroAssembler::xword, memory_order_relaxed);
+
+    // XCHG, memory_order_conservative
     AtomicStubMark mark_xchg_4(_masm, &aarch64_atomic_xchg_4_impl);
     gen_swpal_entry(Assembler::word);
     AtomicStubMark mark_xchg_8_impl(_masm, &aarch64_atomic_xchg_8_impl);
@@ -7131,7 +7377,8 @@ class StubGenerator: public StubCodeGenerator {
 
     // generate GHASH intrinsics code
     if (UseGHASHIntrinsics) {
-      StubRoutines::_ghash_processBlocks = generate_ghash_processBlocks();
+      // StubRoutines::_ghash_processBlocks = generate_ghash_processBlocks();
+      StubRoutines::_ghash_processBlocks = generate_ghash_processBlocks_wide();
     }
 
     if (UseBASE64Intrinsics) {
@@ -7148,6 +7395,8 @@ class StubGenerator: public StubCodeGenerator {
       StubRoutines::_aescrypt_decryptBlock = generate_aescrypt_decryptBlock();
       StubRoutines::_cipherBlockChaining_encryptAESCrypt = generate_cipherBlockChaining_encryptAESCrypt();
       StubRoutines::_cipherBlockChaining_decryptAESCrypt = generate_cipherBlockChaining_decryptAESCrypt();
+      StubRoutines::_galoisCounterMode_AESCrypt = generate_galoisCounterMode_AESCrypt();
+      StubRoutines::_counterMode_AESCrypt = generate_counterMode_AESCrypt();
     }
 
     if (UseSHA1Intrinsics) {
@@ -7213,6 +7462,8 @@ void StubGenerator_generate(CodeBuffer* code, bool all) {
 
 DEFAULT_ATOMIC_OP(fetch_add, 4, )
 DEFAULT_ATOMIC_OP(fetch_add, 8, )
+DEFAULT_ATOMIC_OP(fetch_add, 4, _relaxed)
+DEFAULT_ATOMIC_OP(fetch_add, 8, _relaxed)
 DEFAULT_ATOMIC_OP(xchg, 4, )
 DEFAULT_ATOMIC_OP(xchg, 8, )
 DEFAULT_ATOMIC_OP(cmpxchg, 1, )

src/hotspot/cpu/aarch64/stubRoutines_aarch64.hpp

@@ -36,7 +36,7 @@ static bool    returns_to_call_stub(address return_pc)   {
 
 enum platform_dependent_constants {
   code_size1 = 19000,          // simply increase if too small (assembler will crash if too small)
-  code_size2 = 28000           // simply increase if too small (assembler will crash if too small)
+  code_size2 = 38000           // simply increase if too small (assembler will crash if too small)
 };
 
 class aarch64 {

src/hotspot/cpu/aarch64/vm_version_aarch64.cpp

@@ -237,6 +237,9 @@ void VM_Version::initialize() {
       warning("UseAESIntrinsics enabled, but UseAES not, enabling");
       UseAES = true;
     }
+    if (FLAG_IS_DEFAULT(UseAESCTRIntrinsics)) {
+      FLAG_SET_DEFAULT(UseAESCTRIntrinsics, true);
+    }
   } else {
     if (UseAES) {
       warning("AES instructions are not available on this CPU");
@@ -246,12 +249,12 @@ void VM_Version::initialize() {
       warning("AES intrinsics are not available on this CPU");
       FLAG_SET_DEFAULT(UseAESIntrinsics, false);
     }
+    if (UseAESCTRIntrinsics) {
+      warning("AES/CTR intrinsics are not available on this CPU");
+      FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false);
+    }
   }
 
-  if (UseAESCTRIntrinsics) {
-    warning("AES/CTR intrinsics are not available on this CPU");
-    FLAG_SET_DEFAULT(UseAESCTRIntrinsics, false);
-  }
 
   if (FLAG_IS_DEFAULT(UseCRC32Intrinsics)) {
     UseCRC32Intrinsics = true;

src/hotspot/cpu/arm/frame_arm.cpp

@@ -574,7 +574,6 @@ frame::frame(void* sp, void* fp, void* pc) {
   init((intptr_t*)sp, (intptr_t*)fp, (address)pc);
 }
 
-void frame::pd_ps() {}
 #endif
 
 intptr_t *frame::initial_deoptimization_info() {

src/hotspot/cpu/arm/matcher_arm.hpp

@@ -56,6 +56,9 @@
   // No support for generic vector operands.
   static const bool supports_generic_vector_operands = false;
 
+  // No support for 48 extra htbl entries in aes-gcm intrinsic
+  static const int htbl_entries = -1;
+
   static constexpr bool isSimpleConstant64(jlong value) {
     // Will one (StoreL ConL) be cheaper than two (StoreI ConI)?.
     return false;
@@ -152,4 +155,7 @@
       return false;
   }
 
+  // Implements a variant of EncodeISOArrayNode that encode ASCII only
+  static const bool supports_encode_ascii_array = false;
+
 #endif // CPU_ARM_MATCHER_ARM_HPP

src/hotspot/cpu/arm/stubGenerator_arm.cpp

@@ -635,17 +635,17 @@ class StubGenerator: public StubCodeGenerator {
     Register result_hi = R1;
     Register src       = R0;
 
-    if (!os::is_MP()) {
-      __ ldmia(src, RegisterSet(result_lo, result_hi));
-      __ bx(LR);
-    } else if (VM_Version::supports_ldrexd()) {
+    if (VM_Version::supports_ldrexd()) {
       __ ldrexd(result_lo, Address(src));
       __ clrex(); // FIXME: safe to remove?
-      __ bx(LR);
+    } else if (!os::is_MP()) {
+      // Last-ditch attempt: we are allegedly running on uni-processor.
+      // Load the thing non-atomically and hope for the best.
+      __ ldmia(src, RegisterSet(result_lo, result_hi));
     } else {
       __ stop("Atomic load(jlong) unsupported on this platform");
-      __ bx(LR);
     }
+    __ bx(LR);
 
     return start;
   }
@@ -662,10 +662,7 @@ class StubGenerator: public StubCodeGenerator {
     Register scratch_hi    = R3;  /* After load from stack */
     Register result    = R3;
 
-    if (!os::is_MP()) {
-      __ stmia(dest, RegisterSet(newval_lo, newval_hi));
-      __ bx(LR);
-    } else if (VM_Version::supports_ldrexd()) {
+    if (VM_Version::supports_ldrexd()) {
       __ mov(Rtemp, dest);  // get dest to Rtemp
       Label retry;
       __ bind(retry);
@@ -673,11 +670,14 @@ class StubGenerator: public StubCodeGenerator {
       __ strexd(result, R0, Address(Rtemp));
       __ rsbs(result, result, 1);
       __ b(retry, eq);
-      __ bx(LR);
+    } else if (!os::is_MP()) {
+      // Last-ditch attempt: we are allegedly running on uni-processor.
+      // Store the thing non-atomically and hope for the best.
+      __ stmia(dest, RegisterSet(newval_lo, newval_hi));
     } else {
       __ stop("Atomic store(jlong) unsupported on this platform");
-      __ bx(LR);
     }
+    __ bx(LR);
 
     return start;
   }

src/hotspot/cpu/ppc/assembler_ppc.hpp

@@ -47,6 +47,9 @@ class Address {
   Address(Register b, address d = 0)
     : _base(b), _index(noreg), _disp((intptr_t)d) {}
 
+  Address(Register b, ByteSize d)
+    : _base(b), _index(noreg), _disp((intptr_t)d) {}
+
   Address(Register b, intptr_t d)
     : _base(b), _index(noreg), _disp(d) {}
 

src/hotspot/cpu/ppc/frame_ppc.cpp

@@ -387,5 +387,4 @@ frame::frame(void* sp, void* fp, void* pc) : _sp((intptr_t*)sp), _unextended_sp(
   find_codeblob_and_set_pc_and_deopt_state((address)pc); // also sets _fp and adjusts _unextended_sp
 }
 
-void frame::pd_ps() {}
 #endif

src/hotspot/cpu/ppc/gc/shared/barrierSetAssembler_ppc.cpp

@@ -111,16 +111,28 @@ void BarrierSetAssembler::load_at(MacroAssembler* masm, DecoratorSet decorators,
   }
 }
 
+// Generic implementation. GCs can provide an optimized one.
 void BarrierSetAssembler::resolve_jobject(MacroAssembler* masm, Register value,
                                           Register tmp1, Register tmp2,
                                           MacroAssembler::PreservationLevel preservation_level) {
-  Label done;
+  Label done, not_weak, verify;
   __ cmpdi(CCR0, value, 0);
   __ beq(CCR0, done);         // Use NULL as-is.
 
-  __ clrrdi(tmp1, value, JNIHandles::weak_tag_size);
-  __ ld(value, 0, tmp1);      // Resolve (untagged) jobject.
+  __ andi_(tmp1, value, JNIHandles::weak_tag_mask);
+  __ beq(CCR0, not_weak);     // Test for jweak tag.
 
+  // Resolve (untagged) jobject.
+  __ clrrdi(value, value, JNIHandles::weak_tag_size);
+  load_at(masm, IN_NATIVE | ON_PHANTOM_OOP_REF, T_OBJECT,
+          value, (intptr_t)0, value, tmp1, tmp2, preservation_level);
+  __ b(verify);
+
+  __ bind(not_weak);
+  load_at(masm, IN_NATIVE, T_OBJECT,
+          value, (intptr_t)0, value, tmp1, tmp2, preservation_level);
+
+  __ bind(verify);
   __ verify_oop(value, FILE_AND_LINE);
   __ bind(done);
 }
@@ -178,7 +190,7 @@ void BarrierSetAssembler::c2i_entry_barrier(MacroAssembler *masm, Register tmp1,
   __ ld(tmp1_class_loader_data, in_bytes(InstanceKlass::class_loader_data_offset()), tmp1);
 
   // Fast path: If class loader is strong, the holder cannot be unloaded.
-  __ ld(tmp2, in_bytes(ClassLoaderData::keep_alive_offset()), tmp1_class_loader_data);
+  __ lwz(tmp2, in_bytes(ClassLoaderData::keep_alive_offset()), tmp1_class_loader_data);
   __ cmpdi(CCR0, tmp2, 0);
   __ bne(CCR0, skip_barrier);
 

src/hotspot/cpu/ppc/gc/shared/modRefBarrierSetAssembler_ppc.cpp

@@ -26,6 +26,7 @@
 #include "precompiled.hpp"
 #include "asm/macroAssembler.inline.hpp"
 #include "gc/shared/modRefBarrierSetAssembler.hpp"
+#include "runtime/jniHandles.hpp"
 
 #define __ masm->
 
@@ -74,3 +75,17 @@ void ModRefBarrierSetAssembler::store_at(MacroAssembler* masm, DecoratorSet deco
                                   preservation_level);
   }
 }
+
+void ModRefBarrierSetAssembler::resolve_jobject(MacroAssembler* masm, Register value,
+                                                Register tmp1, Register tmp2,
+                                                MacroAssembler::PreservationLevel preservation_level) {
+  Label done;
+  __ cmpdi(CCR0, value, 0);
+  __ beq(CCR0, done);         // Use NULL as-is.
+
+  __ clrrdi(tmp1, value, JNIHandles::weak_tag_size);
+  __ ld(value, 0, tmp1);      // Resolve (untagged) jobject.
+
+  __ verify_oop(value, FILE_AND_LINE);
+  __ bind(done);
+}

src/hotspot/cpu/ppc/gc/shared/modRefBarrierSetAssembler_ppc.hpp

@@ -57,6 +57,10 @@ public:
                         Register base, RegisterOrConstant ind_or_offs, Register val,
                         Register tmp1, Register tmp2, Register tmp3,
                         MacroAssembler::PreservationLevel preservation_level);
+
+  virtual void resolve_jobject(MacroAssembler* masm, Register value,
+                               Register tmp1, Register tmp2,
+                               MacroAssembler::PreservationLevel preservation_level);
 };
 
 #endif // CPU_PPC_GC_SHARED_MODREFBARRIERSETASSEMBLER_PPC_HPP

src/hotspot/cpu/ppc/gc/z/zBarrierSetAssembler_ppc.cpp

@@ -0,0 +1,567 @@
+/*
+ * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2021 SAP SE. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+#include "asm/register.hpp"
+#include "precompiled.hpp"
+#include "asm/macroAssembler.inline.hpp"
+#include "code/codeBlob.hpp"
+#include "code/vmreg.inline.hpp"
+#include "gc/z/zBarrier.inline.hpp"
+#include "gc/z/zBarrierSet.hpp"
+#include "gc/z/zBarrierSetAssembler.hpp"
+#include "gc/z/zBarrierSetRuntime.hpp"
+#include "gc/z/zThreadLocalData.hpp"
+#include "memory/resourceArea.hpp"
+#include "register_ppc.hpp"
+#include "runtime/sharedRuntime.hpp"
+#include "utilities/globalDefinitions.hpp"
+#include "utilities/macros.hpp"
+#ifdef COMPILER1
+#include "c1/c1_LIRAssembler.hpp"
+#include "c1/c1_MacroAssembler.hpp"
+#include "gc/z/c1/zBarrierSetC1.hpp"
+#endif // COMPILER1
+#ifdef COMPILER2
+#include "gc/z/c2/zBarrierSetC2.hpp"
+#endif // COMPILER2
+
+#undef __
+#define __ masm->
+
+void ZBarrierSetAssembler::load_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
+                                   Register base, RegisterOrConstant ind_or_offs, Register dst,
+                                   Register tmp1, Register tmp2,
+                                   MacroAssembler::PreservationLevel preservation_level, Label *L_handle_null) {
+  __ block_comment("load_at (zgc) {");
+
+  // Check whether a special gc barrier is required for this particular load
+  // (e.g. whether it's a reference load or not)
+  if (!ZBarrierSet::barrier_needed(decorators, type)) {
+    BarrierSetAssembler::load_at(masm, decorators, type, base, ind_or_offs, dst,
+                                 tmp1, tmp2, preservation_level, L_handle_null);
+    return;
+  }
+
+  if (ind_or_offs.is_register()) {
+    assert_different_registers(base, ind_or_offs.as_register(), tmp1, tmp2, R0, noreg);
+    assert_different_registers(dst, ind_or_offs.as_register(), tmp1, tmp2, R0, noreg);
+  } else {
+    assert_different_registers(base, tmp1, tmp2, R0, noreg);
+    assert_different_registers(dst, tmp1, tmp2, R0, noreg);
+  }
+
+  /* ==== Load the pointer using the standard implementation for the actual heap access
+          and the decompression of compressed pointers ==== */
+  // Result of 'load_at' (standard implementation) will be written back to 'dst'.
+  // As 'base' is required for the C-call, it must be reserved in case of a register clash.
+  Register saved_base = base;
+  if (base == dst) {
+    __ mr(tmp2, base);
+    saved_base = tmp2;
+  }
+
+  BarrierSetAssembler::load_at(masm, decorators, type, base, ind_or_offs, dst,
+                               tmp1, noreg, preservation_level, L_handle_null);
+
+  /* ==== Check whether pointer is dirty ==== */
+  Label skip_barrier;
+
+  // Load bad mask into scratch register.
+  __ ld(tmp1, (intptr_t) ZThreadLocalData::address_bad_mask_offset(), R16_thread);
+
+  // The color bits of the to-be-tested pointer do not have to be equivalent to the 'bad_mask' testing bits.
+  // A pointer is classified as dirty if any of the color bits that also match the bad mask is set.
+  // Conversely, it follows that the logical AND of the bad mask and the pointer must be zero
+  // if the pointer is not dirty.
+  // Only dirty pointers must be processed by this barrier, so we can skip it in case the latter condition holds true.
+  __ and_(tmp1, tmp1, dst);
+  __ beq(CCR0, skip_barrier);
+
+  /* ==== Invoke barrier ==== */
+  int nbytes_save = 0;
+
+  const bool needs_frame = preservation_level >= MacroAssembler::PRESERVATION_FRAME_LR;
+  const bool preserve_gp_registers = preservation_level >= MacroAssembler::PRESERVATION_FRAME_LR_GP_REGS;
+  const bool preserve_fp_registers = preservation_level >= MacroAssembler::PRESERVATION_FRAME_LR_GP_FP_REGS;
+
+  const bool preserve_R3 = dst != R3_ARG1;
+
+  if (needs_frame) {
+    if (preserve_gp_registers) {
+      nbytes_save = (preserve_fp_registers
+                     ? MacroAssembler::num_volatile_gp_regs + MacroAssembler::num_volatile_fp_regs
+                     : MacroAssembler::num_volatile_gp_regs) * BytesPerWord;
+      nbytes_save -= preserve_R3 ? 0 : BytesPerWord;
+      __ save_volatile_gprs(R1_SP, -nbytes_save, preserve_fp_registers, preserve_R3);
+    }
+
+    __ save_LR_CR(tmp1);
+    __ push_frame_reg_args(nbytes_save, tmp1);
+  }
+
+  // Setup arguments
+  if (saved_base != R3_ARG1) {
+    __ mr_if_needed(R3_ARG1, dst);
+    __ add(R4_ARG2, ind_or_offs, saved_base);
+  } else if (dst != R4_ARG2) {
+    __ add(R4_ARG2, ind_or_offs, saved_base);
+    __ mr(R3_ARG1, dst);
+  } else {
+    __ add(R0, ind_or_offs, saved_base);
+    __ mr(R3_ARG1, dst);
+    __ mr(R4_ARG2, R0);
+  }
+
+  __ call_VM_leaf(ZBarrierSetRuntime::load_barrier_on_oop_field_preloaded_addr(decorators));
+
+  Register result = R3_RET;
+  if (needs_frame) {
+    __ pop_frame();
+    __ restore_LR_CR(tmp1);
+
+    if (preserve_R3) {
+      __ mr(R0, R3_RET);
+      result = R0;
+    }
+
+    if (preserve_gp_registers) {
+      __ restore_volatile_gprs(R1_SP, -nbytes_save, preserve_fp_registers, preserve_R3);
+    }
+  }
+  __ mr_if_needed(dst, result);
+
+  __ bind(skip_barrier);
+  __ block_comment("} load_at (zgc)");
+}
+
+#ifdef ASSERT
+// The Z store barrier only verifies the pointers it is operating on and is thus a sole debugging measure.
+void ZBarrierSetAssembler::store_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
+                                    Register base, RegisterOrConstant ind_or_offs, Register val,
+                                    Register tmp1, Register tmp2, Register tmp3,
+                                    MacroAssembler::PreservationLevel preservation_level) {
+  __ block_comment("store_at (zgc) {");
+
+  // If the 'val' register is 'noreg', the to-be-stored value is a null pointer.
+  if (is_reference_type(type) && val != noreg) {
+    __ ld(tmp1, in_bytes(ZThreadLocalData::address_bad_mask_offset()), R16_thread);
+    __ and_(tmp1, tmp1, val);
+    __ asm_assert_eq("Detected dirty pointer on the heap in Z store barrier");
+  }
+
+  // Store value
+  BarrierSetAssembler::store_at(masm, decorators, type, base, ind_or_offs, val, tmp1, tmp2, tmp3, preservation_level);
+
+  __ block_comment("} store_at (zgc)");
+}
+#endif // ASSERT
+
+void ZBarrierSetAssembler::arraycopy_prologue(MacroAssembler *masm, DecoratorSet decorators, BasicType component_type,
+                                              Register src, Register dst, Register count,
+                                              Register preserve1, Register preserve2) {
+  __ block_comment("arraycopy_prologue (zgc) {");
+
+  /* ==== Check whether a special gc barrier is required for this particular load ==== */
+  if (!is_reference_type(component_type)) {
+    return;
+  }
+
+  Label skip_barrier;
+
+  // Fast path: Array is of length zero
+  __ cmpdi(CCR0, count, 0);
+  __ beq(CCR0, skip_barrier);
+
+  /* ==== Ensure register sanity ==== */
+  Register tmp_R11 = R11_scratch1;
+
+  assert_different_registers(src, dst, count, tmp_R11, noreg);
+  if (preserve1 != noreg) {
+    // Not technically required, but unlikely being intended.
+    assert_different_registers(preserve1, preserve2);
+  }
+
+  /* ==== Invoke barrier (slowpath) ==== */
+  int nbytes_save = 0;
+
+  {
+    assert(!noreg->is_volatile(), "sanity");
+
+    if (preserve1->is_volatile()) {
+      __ std(preserve1, -BytesPerWord * ++nbytes_save, R1_SP);
+    }
+
+    if (preserve2->is_volatile() && preserve1 != preserve2) {
+      __ std(preserve2, -BytesPerWord * ++nbytes_save, R1_SP);
+    }
+
+    __ std(src, -BytesPerWord * ++nbytes_save, R1_SP);
+    __ std(dst, -BytesPerWord * ++nbytes_save, R1_SP);
+    __ std(count, -BytesPerWord * ++nbytes_save, R1_SP);
+
+    __ save_LR_CR(tmp_R11);
+    __ push_frame_reg_args(nbytes_save, tmp_R11);
+  }
+
+  // ZBarrierSetRuntime::load_barrier_on_oop_array_addr(src, count)
+  if (count == R3_ARG1) {
+    if (src == R4_ARG2) {
+      // Arguments are provided in reverse order
+      __ mr(tmp_R11, count);
+      __ mr(R3_ARG1, src);
+      __ mr(R4_ARG2, tmp_R11);
+    } else {
+      __ mr(R4_ARG2, count);
+      __ mr(R3_ARG1, src);
+    }
+  } else {
+    __ mr_if_needed(R3_ARG1, src);
+    __ mr_if_needed(R4_ARG2, count);
+  }
+
+  __ call_VM_leaf(ZBarrierSetRuntime::load_barrier_on_oop_array_addr());
+
+  __ pop_frame();
+  __ restore_LR_CR(tmp_R11);
+
+  {
+    __ ld(count, -BytesPerWord * nbytes_save--, R1_SP);
+    __ ld(dst, -BytesPerWord * nbytes_save--, R1_SP);
+    __ ld(src, -BytesPerWord * nbytes_save--, R1_SP);
+
+    if (preserve2->is_volatile() && preserve1 != preserve2) {
+      __ ld(preserve2, -BytesPerWord * nbytes_save--, R1_SP);
+    }
+
+    if (preserve1->is_volatile()) {
+      __ ld(preserve1, -BytesPerWord * nbytes_save--, R1_SP);
+    }
+  }
+
+  __ bind(skip_barrier);
+
+  __ block_comment("} arraycopy_prologue (zgc)");
+}
+
+void ZBarrierSetAssembler::try_resolve_jobject_in_native(MacroAssembler* masm, Register dst, Register jni_env,
+                                                         Register obj, Register tmp, Label& slowpath) {
+  __ block_comment("try_resolve_jobject_in_native (zgc) {");
+
+  assert_different_registers(jni_env, obj, tmp);
+
+  // Resolve the pointer using the standard implementation for weak tag handling and pointer verfication.
+  BarrierSetAssembler::try_resolve_jobject_in_native(masm, dst, jni_env, obj, tmp, slowpath);
+
+  // Check whether pointer is dirty.
+  __ ld(tmp,
+        in_bytes(ZThreadLocalData::address_bad_mask_offset() - JavaThread::jni_environment_offset()),
+        jni_env);
+
+  __ and_(tmp, obj, tmp);
+  __ bne(CCR0, slowpath);
+
+  __ block_comment("} try_resolve_jobject_in_native (zgc)");
+}
+
+#undef __
+
+#ifdef COMPILER1
+#define __ ce->masm()->
+
+// Code emitted by LIR node "LIR_OpZLoadBarrierTest" which in turn is emitted by ZBarrierSetC1::load_barrier.
+// The actual compare and branch instructions are represented as stand-alone LIR nodes.
+void ZBarrierSetAssembler::generate_c1_load_barrier_test(LIR_Assembler* ce,
+                                                         LIR_Opr ref) const {
+  __ block_comment("load_barrier_test (zgc) {");
+
+  __ ld(R0, in_bytes(ZThreadLocalData::address_bad_mask_offset()), R16_thread);
+  __ andr(R0, R0, ref->as_pointer_register());
+  __ cmpdi(CCR5 /* as mandated by LIR node */, R0, 0);
+
+  __ block_comment("} load_barrier_test (zgc)");
+}
+
+// Code emitted by code stub "ZLoadBarrierStubC1" which in turn is emitted by ZBarrierSetC1::load_barrier.
+// Invokes the runtime stub which is defined just below.
+void ZBarrierSetAssembler::generate_c1_load_barrier_stub(LIR_Assembler* ce,
+                                                         ZLoadBarrierStubC1* stub) const {
+  __ block_comment("c1_load_barrier_stub (zgc) {");
+
+  __ bind(*stub->entry());
+
+  /* ==== Determine relevant data registers and ensure register sanity ==== */
+  Register ref = stub->ref()->as_register();
+  Register ref_addr = noreg;
+
+  // Determine reference address
+  if (stub->tmp()->is_valid()) {
+    // 'tmp' register is given, so address might have an index or a displacement.
+    ce->leal(stub->ref_addr(), stub->tmp());
+    ref_addr = stub->tmp()->as_pointer_register();
+  } else {
+    // 'tmp' register is not given, so address must have neither an index nor a displacement.
+    // The address' base register is thus usable as-is.
+    assert(stub->ref_addr()->as_address_ptr()->disp() == 0, "illegal displacement");
+    assert(!stub->ref_addr()->as_address_ptr()->index()->is_valid(), "illegal index");
+
+    ref_addr = stub->ref_addr()->as_address_ptr()->base()->as_pointer_register();
+  }
+
+  assert_different_registers(ref, ref_addr, R0, noreg);
+
+  /* ==== Invoke stub ==== */
+  // Pass arguments via stack. The stack pointer will be bumped by the stub.
+  __ std(ref, (intptr_t) -1 * BytesPerWord, R1_SP);
+  __ std(ref_addr, (intptr_t) -2 * BytesPerWord, R1_SP);
+
+  __ load_const_optimized(R0, stub->runtime_stub());
+  __ call_stub(R0);
+
+  // The runtime stub passes the result via the R0 register, overriding the previously-loaded stub address.
+  __ mr_if_needed(ref, R0);
+  __ b(*stub->continuation());
+
+  __ block_comment("} c1_load_barrier_stub (zgc)");
+}
+
+#undef __
+#define __ sasm->
+
+// Code emitted by runtime code stub which in turn is emitted by ZBarrierSetC1::generate_c1_runtime_stubs.
+void ZBarrierSetAssembler::generate_c1_load_barrier_runtime_stub(StubAssembler* sasm,
+                                                                 DecoratorSet decorators) const {
+  __ block_comment("c1_load_barrier_runtime_stub (zgc) {");
+
+  const int stack_parameters = 2;
+  const int nbytes_save = (MacroAssembler::num_volatile_regs + stack_parameters) * BytesPerWord;
+
+  __ save_volatile_gprs(R1_SP, -nbytes_save);
+  __ save_LR_CR(R0);
+
+  // Load arguments back again from the stack.
+  __ ld(R3_ARG1, (intptr_t) -1 * BytesPerWord, R1_SP); // ref
+  __ ld(R4_ARG2, (intptr_t) -2 * BytesPerWord, R1_SP); // ref_addr
+
+  __ push_frame_reg_args(nbytes_save, R0);
+
+  __ call_VM_leaf(ZBarrierSetRuntime::load_barrier_on_oop_field_preloaded_addr(decorators));
+
+  __ verify_oop(R3_RET, "Bad pointer after barrier invocation");
+  __ mr(R0, R3_RET);
+
+  __ pop_frame();
+  __ restore_LR_CR(R3_RET);
+  __ restore_volatile_gprs(R1_SP, -nbytes_save);
+
+  __ blr();
+
+  __ block_comment("} c1_load_barrier_runtime_stub (zgc)");
+}
+
+#undef __
+#endif // COMPILER1
+
+#ifdef COMPILER2
+
+OptoReg::Name ZBarrierSetAssembler::refine_register(const Node* node, OptoReg::Name opto_reg) const {
+  if (!OptoReg::is_reg(opto_reg)) {
+    return OptoReg::Bad;
+  }
+
+  VMReg vm_reg = OptoReg::as_VMReg(opto_reg);
+  if ((vm_reg->is_Register() || vm_reg ->is_FloatRegister()) && (opto_reg & 1) != 0) {
+    return OptoReg::Bad;
+  }
+
+  return opto_reg;
+}
+
+#define __ _masm->
+
+class ZSaveLiveRegisters {
+
+ private:
+  MacroAssembler* _masm;
+  RegMask _reg_mask;
+  Register _result_reg;
+
+ public:
+  ZSaveLiveRegisters(MacroAssembler *masm, ZLoadBarrierStubC2 *stub)
+      : _masm(masm), _reg_mask(stub->live()), _result_reg(stub->ref()) {
+
+    const int total_regs_amount = iterate_over_register_mask(ACTION_SAVE);
+
+    __ save_LR_CR(R0);
+    __ push_frame_reg_args(total_regs_amount * BytesPerWord, R0);
+  }
+
+  ~ZSaveLiveRegisters() {
+    __ pop_frame();
+    __ restore_LR_CR(R0);
+
+    iterate_over_register_mask(ACTION_RESTORE);
+  }
+
+ private:
+  enum IterationAction : int {
+    ACTION_SAVE = 0,
+    ACTION_RESTORE = 1
+  };
+
+  int iterate_over_register_mask(IterationAction action) {
+    int reg_save_index = 0;
+    RegMaskIterator live_regs_iterator(_reg_mask);
+
+    while(live_regs_iterator.has_next()) {
+      const OptoReg::Name opto_reg = live_regs_iterator.next();
+
+      // Filter out stack slots (spilled registers, i.e., stack-allocated registers).
+      if (!OptoReg::is_reg(opto_reg)) {
+        continue;
+      }
+
+      const VMReg vm_reg = OptoReg::as_VMReg(opto_reg);
+      if (vm_reg->is_Register()) {
+        Register std_reg = vm_reg->as_Register();
+
+        // '_result_reg' will hold the end result of the operation. Its content must thus not be preserved.
+        if (std_reg == _result_reg) {
+          continue;
+        }
+
+        if (std_reg->encoding() >= R2->encoding() && std_reg->encoding() <= R12->encoding()) {
+          reg_save_index++;
+
+          if (action == ACTION_SAVE) {
+            _masm->std(std_reg, (intptr_t) -reg_save_index * BytesPerWord, R1_SP);
+          } else if (action == ACTION_RESTORE) {
+            _masm->ld(std_reg, (intptr_t) -reg_save_index * BytesPerWord, R1_SP);
+          } else {
+            fatal("Sanity");
+          }
+        }
+      } else if (vm_reg->is_FloatRegister()) {
+        FloatRegister fp_reg = vm_reg->as_FloatRegister();
+        if (fp_reg->encoding() >= F0->encoding() && fp_reg->encoding() <= F13->encoding()) {
+          reg_save_index++;
+
+          if (action == ACTION_SAVE) {
+            _masm->stfd(fp_reg, (intptr_t) -reg_save_index * BytesPerWord, R1_SP);
+          } else if (action == ACTION_RESTORE) {
+            _masm->lfd(fp_reg, (intptr_t) -reg_save_index * BytesPerWord, R1_SP);
+          } else {
+            fatal("Sanity");
+          }
+        }
+      } else if (vm_reg->is_ConditionRegister()) {
+        // NOP. Conditions registers are covered by save_LR_CR
+      } else {
+        if (vm_reg->is_VectorRegister()) {
+          fatal("Vector registers are unsupported. Found register %s", vm_reg->name());
+        } else if (vm_reg->is_SpecialRegister()) {
+          fatal("Special registers are unsupported. Found register %s", vm_reg->name());
+        } else {
+          fatal("Register type is not known");
+        }
+      }
+    }
+
+    return reg_save_index;
+  }
+};
+
+#undef __
+#define __ _masm->
+
+class ZSetupArguments {
+ private:
+  MacroAssembler* const _masm;
+  const Register        _ref;
+  const Address         _ref_addr;
+
+ public:
+  ZSetupArguments(MacroAssembler* masm, ZLoadBarrierStubC2* stub) :
+      _masm(masm),
+      _ref(stub->ref()),
+      _ref_addr(stub->ref_addr()) {
+
+    // Desired register/argument configuration:
+    // _ref: R3_ARG1
+    // _ref_addr: R4_ARG2
+
+    // '_ref_addr' can be unspecified. In that case, the barrier will not heal the reference.
+    if (_ref_addr.base() == noreg) {
+      assert_different_registers(_ref, R0, noreg);
+
+      __ mr_if_needed(R3_ARG1, _ref);
+      __ li(R4_ARG2, 0);
+    } else {
+      assert_different_registers(_ref, _ref_addr.base(), R0, noreg);
+      assert(!_ref_addr.index()->is_valid(), "reference addresses must not contain an index component");
+
+      if (_ref != R4_ARG2) {
+        // Calculate address first as the address' base register might clash with R4_ARG2
+        __ add(R4_ARG2, (intptr_t) _ref_addr.disp(), _ref_addr.base());
+        __ mr_if_needed(R3_ARG1, _ref);
+      } else if (_ref_addr.base() != R3_ARG1) {
+        __ mr(R3_ARG1, _ref);
+        __ add(R4_ARG2, (intptr_t) _ref_addr.disp(), _ref_addr.base()); // Cloberring _ref
+      } else {
+        // Arguments are provided in inverse order (i.e. _ref == R4_ARG2, _ref_addr == R3_ARG1)
+        __ mr(R0, _ref);
+        __ add(R4_ARG2, (intptr_t) _ref_addr.disp(), _ref_addr.base());
+        __ mr(R3_ARG1, R0);
+      }
+    }
+  }
+};
+
+#undef __
+#define __ masm->
+
+void ZBarrierSetAssembler::generate_c2_load_barrier_stub(MacroAssembler* masm, ZLoadBarrierStubC2* stub) const {
+  __ block_comment("generate_c2_load_barrier_stub (zgc) {");
+
+  __ bind(*stub->entry());
+
+  Register ref = stub->ref();
+  Address ref_addr = stub->ref_addr();
+
+  assert_different_registers(ref, ref_addr.base());
+
+  {
+    ZSaveLiveRegisters save_live_registers(masm, stub);
+    ZSetupArguments setup_arguments(masm, stub);
+
+    __ call_VM_leaf(stub->slow_path());
+    __ mr_if_needed(ref, R3_RET);
+  }
+
+  __ b(*stub->continuation());
+
+  __ block_comment("} generate_c2_load_barrier_stub (zgc)");
+}
+
+#undef __
+#endif // COMPILER2

src/hotspot/cpu/ppc/gc/z/zBarrierSetAssembler_ppc.hpp

@@ -0,0 +1,86 @@
+/*
+ * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2021 SAP SE. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+#ifndef CPU_PPC_GC_Z_ZBARRIERSETASSEMBLER_PPC_HPP
+#define CPU_PPC_GC_Z_ZBARRIERSETASSEMBLER_PPC_HPP
+
+#include "code/vmreg.hpp"
+#include "oops/accessDecorators.hpp"
+#ifdef COMPILER2
+#include "opto/optoreg.hpp"
+#endif // COMPILER2
+
+#ifdef COMPILER1
+class LIR_Assembler;
+class LIR_OprDesc;
+typedef LIR_OprDesc* LIR_Opr;
+class StubAssembler;
+class ZLoadBarrierStubC1;
+#endif // COMPILER1
+
+#ifdef COMPILER2
+class Node;
+class ZLoadBarrierStubC2;
+#endif // COMPILER2
+
+class ZBarrierSetAssembler : public ZBarrierSetAssemblerBase {
+public:
+  virtual void load_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
+                       Register base, RegisterOrConstant ind_or_offs, Register dst,
+                       Register tmp1, Register tmp2,
+                       MacroAssembler::PreservationLevel preservation_level, Label *L_handle_null = NULL);
+
+#ifdef ASSERT
+  virtual void store_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
+                        Register base, RegisterOrConstant ind_or_offs, Register val,
+                        Register tmp1, Register tmp2, Register tmp3,
+                        MacroAssembler::PreservationLevel preservation_level);
+#endif // ASSERT
+
+  virtual void arraycopy_prologue(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
+                                  Register src, Register dst, Register count,
+                                  Register preserve1, Register preserve2);
+
+  virtual void try_resolve_jobject_in_native(MacroAssembler* masm, Register dst, Register jni_env,
+                                             Register obj, Register tmp, Label& slowpath);
+
+#ifdef COMPILER1
+  void generate_c1_load_barrier_test(LIR_Assembler* ce,
+                                     LIR_Opr ref) const;
+
+  void generate_c1_load_barrier_stub(LIR_Assembler* ce,
+                                     ZLoadBarrierStubC1* stub) const;
+
+  void generate_c1_load_barrier_runtime_stub(StubAssembler* sasm,
+                                             DecoratorSet decorators) const;
+#endif // COMPILER1
+
+#ifdef COMPILER2
+  OptoReg::Name refine_register(const Node* node, OptoReg::Name opto_reg) const;
+
+  void generate_c2_load_barrier_stub(MacroAssembler* masm, ZLoadBarrierStubC2* stub) const;
+#endif // COMPILER2
+};
+
+#endif // CPU_AARCH64_GC_Z_ZBARRIERSETASSEMBLER_AARCH64_HPP

src/hotspot/cpu/ppc/gc/z/zGlobals_ppc.cpp

@@ -0,0 +1,203 @@
+/*
+ * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2021 SAP SE. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+#include "precompiled.hpp"
+#include "gc/shared/gcLogPrecious.hpp"
+#include "gc/shared/gc_globals.hpp"
+#include "gc/z/zGlobals.hpp"
+#include "runtime/globals.hpp"
+#include "runtime/os.hpp"
+#include "utilities/globalDefinitions.hpp"
+#include "utilities/powerOfTwo.hpp"
+#include <cstddef>
+
+#ifdef LINUX
+#include <sys/mman.h>
+#endif // LINUX
+
+//
+// The overall memory layouts across different power platforms are similar and only differ with regards to
+// the position of the highest addressable bit; the position of the metadata bits and the size of the actual
+// addressable heap address space are adjusted accordingly.
+//
+// The following memory schema shows an exemplary layout in which bit '45' is the highest addressable bit.
+// It is assumed that this virtual memroy address space layout is predominant on the power platform.
+//
+// Standard Address Space & Pointer Layout
+// ---------------------------------------
+//
+//  +--------------------------------+ 0x00007FFFFFFFFFFF (127 TiB - 1)
+//  .                                .
+//  .                                .
+//  .                                .
+//  +--------------------------------+ 0x0000140000000000 (20 TiB)
+//  |         Remapped View          |
+//  +--------------------------------+ 0x0000100000000000 (16 TiB)
+//  .                                .
+//  +--------------------------------+ 0x00000c0000000000 (12 TiB)
+//  |         Marked1 View           |
+//  +--------------------------------+ 0x0000080000000000 (8  TiB)
+//  |         Marked0 View           |
+//  +--------------------------------+ 0x0000040000000000 (4  TiB)
+//  .                                .
+//  +--------------------------------+ 0x0000000000000000
+//
+//   6                  4 4  4 4
+//   3                  6 5  2 1                                             0
+//  +--------------------+----+-----------------------------------------------+
+//  |00000000 00000000 00|1111|11 11111111 11111111 11111111 11111111 11111111|
+//  +--------------------+----+-----------------------------------------------+
+//  |                    |    |
+//  |                    |    * 41-0 Object Offset (42-bits, 4TB address space)
+//  |                    |
+//  |                    * 45-42 Metadata Bits (4-bits)  0001 = Marked0      (Address view 4-8TB)
+//  |                                                    0010 = Marked1      (Address view 8-12TB)
+//  |                                                    0100 = Remapped     (Address view 16-20TB)
+//  |                                                    1000 = Finalizable  (Address view N/A)
+//  |
+//  * 63-46 Fixed (18-bits, always zero)
+//
+
+// Maximum value as per spec (Power ISA v2.07): 2 ^ 60 bytes, i.e. 1 EiB (exbibyte)
+static const unsigned int MAXIMUM_MAX_ADDRESS_BIT = 60;
+
+// Most modern power processors provide an address space with not more than 45 bit addressable bit,
+// that is an address space of 32 TiB in size.
+static const unsigned int DEFAULT_MAX_ADDRESS_BIT = 45;
+
+// Minimum value returned, if probing fails: 64 GiB
+static const unsigned int MINIMUM_MAX_ADDRESS_BIT = 36;
+
+// Determines the highest addressable bit of the virtual address space (depends on platform)
+// by trying to interact with memory in that address range,
+// i.e. by syncing existing mappings (msync) or by temporarily mapping the memory area (mmap).
+// If one of those operations succeeds, it is proven that the targeted memory area is within the virtual address space.
+//
+// To reduce the number of required system calls to a bare minimum, the DEFAULT_MAX_ADDRESS_BIT is intentionally set
+// lower than what the ABI would theoretically permit.
+// Such an avoidance strategy, however, might impose unnecessary limits on processors that exceed this limit.
+// If DEFAULT_MAX_ADDRESS_BIT is addressable, the next higher bit will be tested as well to ensure that
+// the made assumption does not artificially restrict the memory availability.
+static unsigned int probe_valid_max_address_bit(size_t init_bit, size_t min_bit) {
+  assert(init_bit >= min_bit, "Sanity");
+  assert(init_bit <= MAXIMUM_MAX_ADDRESS_BIT, "Test bit is outside the assumed address space range");
+
+#ifdef LINUX
+  unsigned int max_valid_address_bit = 0;
+  void* last_allocatable_address = nullptr;
+
+  const unsigned int page_size = os::vm_page_size();
+
+  for (size_t i = init_bit; i >= min_bit; --i) {
+    void* base_addr = (void*) (((unsigned long) 1U) << i);
+
+    /* ==== Try msync-ing already mapped memory page ==== */
+    if (msync(base_addr, page_size, MS_ASYNC) == 0) {
+      // The page of the given address was synced by the linux kernel and must thus be both, mapped and valid.
+      max_valid_address_bit = i;
+      break;
+    }
+    if (errno != ENOMEM) {
+      // An unexpected error occurred, i.e. an error not indicating that the targeted memory page is unmapped,
+      // but pointing out another type of issue.
+      // Even though this should never happen, those issues may come up due to undefined behavior.
+#ifdef ASSERT
+      fatal("Received '%s' while probing the address space for the highest valid bit", os::errno_name(errno));
+#else // ASSERT
+      log_warning_p(gc)("Received '%s' while probing the address space for the highest valid bit", os::errno_name(errno));
+#endif // ASSERT
+      continue;
+    }
+
+    /* ==== Try mapping memory page on our own ==== */
+    last_allocatable_address = mmap(base_addr, page_size, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0);
+    if (last_allocatable_address != MAP_FAILED) {
+      munmap(last_allocatable_address, page_size);
+    }
+
+    if (last_allocatable_address == base_addr) {
+      // As the linux kernel mapped exactly the page we have requested, the address must be valid.
+      max_valid_address_bit = i;
+      break;
+    }
+
+    log_info_p(gc, init)("Probe failed for bit '%zu'", i);
+  }
+
+  if (max_valid_address_bit == 0) {
+    // Probing did not bring up any usable address bit.
+    // As an alternative, the VM evaluates the address returned by mmap as it is expected that the reserved page
+    // will be close to the probed address that was out-of-range.
+    // As per mmap(2), "the kernel [will take] [the address] as a hint about where to
+    // place the mapping; on Linux, the mapping will be created at a nearby page boundary".
+    // It should thus be a "close enough" approximation to the real virtual memory address space limit.
+    //
+    // This recovery strategy is only applied in production builds.
+    // In debug builds, an assertion in 'ZPlatformAddressOffsetBits' will bail out the VM to indicate that
+    // the assumed address space is no longer up-to-date.
+    if (last_allocatable_address != MAP_FAILED) {
+      const unsigned int bitpos = BitsPerSize_t - count_leading_zeros((size_t) last_allocatable_address) - 1;
+      log_info_p(gc, init)("Did not find any valid addresses within the range, using address '%u' instead", bitpos);
+      return bitpos;
+    }
+
+#ifdef ASSERT
+    fatal("Available address space can not be determined");
+#else // ASSERT
+    log_warning_p(gc)("Cannot determine available address space. Falling back to default value.");
+    return DEFAULT_MAX_ADDRESS_BIT;
+#endif // ASSERT
+  } else {
+    if (max_valid_address_bit == init_bit) {
+      // An usable address bit has been found immediately.
+      // To ensure that the entire virtual address space is exploited, the next highest bit will be tested as well.
+      log_info_p(gc, init)("Hit valid address '%u' on first try, retrying with next higher bit", max_valid_address_bit);
+      return MAX2(max_valid_address_bit, probe_valid_max_address_bit(init_bit + 1, init_bit + 1));
+    }
+  }
+
+  log_info_p(gc, init)("Found valid address '%u'", max_valid_address_bit);
+  return max_valid_address_bit;
+#else // LINUX
+  return DEFAULT_MAX_ADDRESS_BIT;
+#endif // LINUX
+}
+
+size_t ZPlatformAddressOffsetBits() {
+  const static unsigned int valid_max_address_offset_bits =
+      probe_valid_max_address_bit(DEFAULT_MAX_ADDRESS_BIT, MINIMUM_MAX_ADDRESS_BIT) + 1;
+  assert(valid_max_address_offset_bits >= MINIMUM_MAX_ADDRESS_BIT,
+         "Highest addressable bit is outside the assumed address space range");
+
+  const size_t max_address_offset_bits = valid_max_address_offset_bits - 3;
+  const size_t min_address_offset_bits = max_address_offset_bits - 2;
+  const size_t address_offset = round_up_power_of_2(MaxHeapSize * ZVirtualToPhysicalRatio);
+  const size_t address_offset_bits = log2i_exact(address_offset);
+
+  return clamp(address_offset_bits, min_address_offset_bits, max_address_offset_bits);
+}
+
+size_t ZPlatformAddressMetadataShift() {
+  return ZPlatformAddressOffsetBits();
+}

src/hotspot/cpu/ppc/gc/z/zGlobals_ppc.hpp

@@ -0,0 +1,36 @@
+/*
+ * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2021 SAP SE. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+#ifndef CPU_PPC_GC_Z_ZGLOBALS_PPC_HPP
+#define CPU_PPC_GC_Z_ZGLOBALS_PPC_HPP
+
+#include "globalDefinitions_ppc.hpp"
+const size_t ZPlatformGranuleSizeShift = 21; // 2MB
+const size_t ZPlatformHeapViews        = 3;
+const size_t ZPlatformCacheLineSize    = DEFAULT_CACHE_LINE_SIZE;
+
+size_t ZPlatformAddressOffsetBits();
+size_t ZPlatformAddressMetadataShift();
+
+#endif // CPU_PPC_GC_Z_ZGLOBALS_PPC_HPP

src/hotspot/cpu/ppc/gc/z/z_ppc.ad

@@ -0,0 +1,298 @@
+//
+// Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+// Copyright (c) 2021 SAP SE. All rights reserved.
+// DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+//
+// This code is free software; you can redistribute it and/or modify it
+// under the terms of the GNU General Public License version 2 only, as
+// published by the Free Software Foundation.
+//
+// This code is distributed in the hope that it will be useful, but WITHOUT
+// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+// version 2 for more details (a copy is included in the LICENSE file that
+// accompanied this code).
+//
+// You should have received a copy of the GNU General Public License version
+// 2 along with this work; if not, write to the Free Software Foundation,
+// Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+//
+// Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+// or visit www.oracle.com if you need additional information or have any
+// questions.
+//
+
+source_hpp %{
+
+#include "gc/shared/gc_globals.hpp"
+#include "gc/z/c2/zBarrierSetC2.hpp"
+#include "gc/z/zThreadLocalData.hpp"
+
+%}
+
+source %{
+
+static void z_load_barrier(MacroAssembler& _masm, const MachNode* node, Address ref_addr, Register ref,
+                           Register tmp, uint8_t barrier_data) {
+  if (barrier_data == ZLoadBarrierElided) {
+    return;
+  }
+
+  ZLoadBarrierStubC2* const stub = ZLoadBarrierStubC2::create(node, ref_addr, ref, tmp, barrier_data);
+  __ ld(tmp, in_bytes(ZThreadLocalData::address_bad_mask_offset()), R16_thread);
+  __ and_(tmp, tmp, ref);
+  __ bne_far(CCR0, *stub->entry(), MacroAssembler::bc_far_optimize_on_relocate);
+  __ bind(*stub->continuation());
+}
+
+static void z_load_barrier_slow_path(MacroAssembler& _masm, const MachNode* node, Address ref_addr, Register ref,
+                                     Register tmp) {
+  ZLoadBarrierStubC2* const stub = ZLoadBarrierStubC2::create(node, ref_addr, ref, tmp, ZLoadBarrierStrong);
+  __ b(*stub->entry());
+  __ bind(*stub->continuation());
+}
+
+static void z_compare_and_swap(MacroAssembler& _masm, const MachNode* node,
+                              Register res, Register mem, Register oldval, Register newval,
+                              Register tmp_xchg, Register tmp_mask,
+                              bool weak, bool acquire) {
+  // z-specific load barrier requires strong CAS operations.
+  // Weak CAS operations are thus only emitted if the barrier is elided.
+  __ cmpxchgd(CCR0, tmp_xchg, oldval, newval, mem,
+              MacroAssembler::MemBarNone, MacroAssembler::cmpxchgx_hint_atomic_update(), res, NULL, true,
+              weak && node->barrier_data() == ZLoadBarrierElided);
+
+  if (node->barrier_data() != ZLoadBarrierElided) {
+    Label skip_barrier;
+
+    __ ld(tmp_mask, in_bytes(ZThreadLocalData::address_bad_mask_offset()), R16_thread);
+    __ and_(tmp_mask, tmp_mask, tmp_xchg);
+    __ beq(CCR0, skip_barrier);
+
+    // CAS must have failed because pointer in memory is bad.
+    z_load_barrier_slow_path(_masm, node, Address(mem), tmp_xchg, res /* used as tmp */);
+
+    __ cmpxchgd(CCR0, tmp_xchg, oldval, newval, mem,
+                MacroAssembler::MemBarNone, MacroAssembler::cmpxchgx_hint_atomic_update(), res, NULL, true, weak);
+
+    __ bind(skip_barrier);
+  }
+
+  if (acquire) {
+    if (support_IRIW_for_not_multiple_copy_atomic_cpu) {
+      // Uses the isync instruction as an acquire barrier.
+      // This exploits the compare and the branch in the z load barrier (load, compare and branch, isync).
+      __ isync();
+    } else {
+      __ sync();
+    }
+  }
+}
+
+static void z_compare_and_exchange(MacroAssembler& _masm, const MachNode* node,
+                                   Register res, Register mem, Register oldval, Register newval, Register tmp,
+                                   bool weak, bool acquire) {
+  // z-specific load barrier requires strong CAS operations.
+  // Weak CAS operations are thus only emitted if the barrier is elided.
+  __ cmpxchgd(CCR0, res, oldval, newval, mem,
+              MacroAssembler::MemBarNone, MacroAssembler::cmpxchgx_hint_atomic_update(), noreg, NULL, true,
+              weak && node->barrier_data() == ZLoadBarrierElided);
+
+  if (node->barrier_data() != ZLoadBarrierElided) {
+    Label skip_barrier;
+    __ ld(tmp, in_bytes(ZThreadLocalData::address_bad_mask_offset()), R16_thread);
+    __ and_(tmp, tmp, res);
+    __ beq(CCR0, skip_barrier);
+
+    z_load_barrier_slow_path(_masm, node, Address(mem), res, tmp);
+
+    __ cmpxchgd(CCR0, res, oldval, newval, mem,
+                MacroAssembler::MemBarNone, MacroAssembler::cmpxchgx_hint_atomic_update(), noreg, NULL, true, weak);
+
+    __ bind(skip_barrier);
+  }
+
+  if (acquire) {
+    if (support_IRIW_for_not_multiple_copy_atomic_cpu) {
+      // Uses the isync instruction as an acquire barrier.
+      // This exploits the compare and the branch in the z load barrier (load, compare and branch, isync).
+      __ isync();
+    } else {
+      __ sync();
+    }
+  }
+}
+
+%}
+
+instruct zLoadP(iRegPdst dst, memoryAlg4 mem, iRegPdst tmp, flagsRegCR0 cr0)
+%{
+  match(Set dst (LoadP mem));
+  effect(TEMP_DEF dst, TEMP tmp, KILL cr0);
+  ins_cost(MEMORY_REF_COST);
+
+  predicate((UseZGC && n->as_Load()->barrier_data() != 0)
+            && (n->as_Load()->is_unordered() || followed_by_acquire(n)));
+
+  format %{ "LD $dst, $mem" %}
+  ins_encode %{
+    assert($mem$$index == 0, "sanity");
+    __ ld($dst$$Register, $mem$$disp, $mem$$base$$Register);
+    z_load_barrier(_masm, this, Address($mem$$base$$Register, $mem$$disp), $dst$$Register, $tmp$$Register, barrier_data());
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+// Load Pointer Volatile
+instruct zLoadP_acq(iRegPdst dst, memoryAlg4 mem, iRegPdst tmp, flagsRegCR0 cr0)
+%{
+  match(Set dst (LoadP mem));
+  effect(TEMP_DEF dst, TEMP tmp, KILL cr0);
+  ins_cost(3 * MEMORY_REF_COST);
+
+  // Predicate on instruction order is implicitly present due to the predicate of the cheaper zLoadP operation
+  predicate(UseZGC && n->as_Load()->barrier_data() != 0);
+
+  format %{ "LD acq $dst, $mem" %}
+  ins_encode %{
+    __ ld($dst$$Register, $mem$$disp, $mem$$base$$Register);
+    z_load_barrier(_masm, this, Address($mem$$base$$Register, $mem$$disp), $dst$$Register, $tmp$$Register, barrier_data());
+
+    // Uses the isync instruction as an acquire barrier.
+    // This exploits the compare and the branch in the z load barrier (load, compare and branch, isync).
+    __ isync();
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndSwapP(iRegIdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                          iRegPdst tmp_xchg, iRegPdst tmp_mask, flagsRegCR0 cr0) %{
+  match(Set res (CompareAndSwapP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp_xchg, TEMP tmp_mask, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && (((CompareAndSwapNode*)n)->order() != MemNode::acquire && ((CompareAndSwapNode*) n)->order() != MemNode::seqcst));
+
+  format %{ "CMPXCHG $res, $mem, $oldval, $newval; as bool; ptr" %}
+  ins_encode %{
+    z_compare_and_swap(_masm, this,
+                       $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register,
+                       $tmp_xchg$$Register, $tmp_mask$$Register,
+                       false /* weak */, false /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndSwapP_acq(iRegIdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                              iRegPdst tmp_xchg, iRegPdst tmp_mask, flagsRegCR0 cr0) %{
+  match(Set res (CompareAndSwapP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp_xchg, TEMP tmp_mask, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && (((CompareAndSwapNode*)n)->order() == MemNode::acquire || ((CompareAndSwapNode*) n)->order() == MemNode::seqcst));
+
+  format %{ "CMPXCHG acq $res, $mem, $oldval, $newval; as bool; ptr" %}
+  ins_encode %{
+    z_compare_and_swap(_masm, this,
+                       $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register,
+                       $tmp_xchg$$Register, $tmp_mask$$Register,
+                       false /* weak */, true /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndSwapPWeak(iRegIdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                              iRegPdst tmp_xchg, iRegPdst tmp_mask, flagsRegCR0 cr0) %{
+  match(Set res (WeakCompareAndSwapP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp_xchg, TEMP tmp_mask, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && ((CompareAndSwapNode*)n)->order() != MemNode::acquire && ((CompareAndSwapNode*) n)->order() != MemNode::seqcst);
+
+  format %{ "weak CMPXCHG $res, $mem, $oldval, $newval; as bool; ptr" %}
+  ins_encode %{
+    z_compare_and_swap(_masm, this,
+                       $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register,
+                       $tmp_xchg$$Register, $tmp_mask$$Register,
+                       true /* weak */, false /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndSwapPWeak_acq(iRegIdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                                  iRegPdst tmp_xchg, iRegPdst tmp_mask, flagsRegCR0 cr0) %{
+  match(Set res (WeakCompareAndSwapP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp_xchg, TEMP tmp_mask, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && (((CompareAndSwapNode*)n)->order() == MemNode::acquire || ((CompareAndSwapNode*) n)->order() == MemNode::seqcst));
+
+  format %{ "weak CMPXCHG acq $res, $mem, $oldval, $newval; as bool; ptr" %}
+  ins_encode %{
+    z_compare_and_swap(_masm, this,
+                       $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register,
+                       $tmp_xchg$$Register, $tmp_mask$$Register,
+                       true /* weak */, true /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndExchangeP(iRegPdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                              iRegPdst tmp, flagsRegCR0 cr0) %{
+  match(Set res (CompareAndExchangeP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && (
+              ((CompareAndSwapNode*)n)->order() != MemNode::acquire
+              && ((CompareAndSwapNode*)n)->order() != MemNode::seqcst
+            ));
+
+  format %{ "CMPXCHG $res, $mem, $oldval, $newval; as ptr; ptr" %}
+  ins_encode %{
+    z_compare_and_exchange(_masm, this,
+                           $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register, $tmp$$Register,
+                           false /* weak */, false /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zCompareAndExchangeP_acq(iRegPdst res, iRegPdst mem, iRegPsrc oldval, iRegPsrc newval,
+                                  iRegPdst tmp, flagsRegCR0 cr0) %{
+  match(Set res (CompareAndExchangeP mem (Binary oldval newval)));
+  effect(TEMP_DEF res, TEMP tmp, KILL cr0);
+
+  predicate((UseZGC && n->as_LoadStore()->barrier_data() == ZLoadBarrierStrong)
+            && (
+              ((CompareAndSwapNode*)n)->order() == MemNode::acquire
+              || ((CompareAndSwapNode*)n)->order() == MemNode::seqcst
+            ));
+
+  format %{ "CMPXCHG acq $res, $mem, $oldval, $newval; as ptr; ptr" %}
+  ins_encode %{
+    z_compare_and_exchange(_masm, this,
+                           $res$$Register, $mem$$Register, $oldval$$Register, $newval$$Register, $tmp$$Register,
+                           false /* weak */, true /* acquire */);
+  %}
+  ins_pipe(pipe_class_default);
+%}
+
+instruct zGetAndSetP(iRegPdst res, iRegPdst mem, iRegPsrc newval, iRegPdst tmp, flagsRegCR0 cr0) %{
+  match(Set res (GetAndSetP mem newval));
+  effect(TEMP_DEF res, TEMP tmp, KILL cr0);
+
+  predicate(UseZGC && n->as_LoadStore()->barrier_data() != 0);
+
+  format %{ "GetAndSetP $res, $mem, $newval" %}
+  ins_encode %{
+    __ getandsetd($res$$Register, $newval$$Register, $mem$$Register, MacroAssembler::cmpxchgx_hint_atomic_update());
+    z_load_barrier(_masm, this, Address(noreg, (intptr_t) 0), $res$$Register, $tmp$$Register, barrier_data());
+
+    if (support_IRIW_for_not_multiple_copy_atomic_cpu) {
+      __ isync();
+    } else {
+      __ sync();
+    }
+  %}
+  ins_pipe(pipe_class_default);
+%}

src/hotspot/cpu/ppc/matcher_ppc.hpp

@@ -57,6 +57,9 @@
   // No support for generic vector operands.
   static const bool supports_generic_vector_operands = false;
 
+  // No support for 48 extra htbl entries in aes-gcm intrinsic
+  static const int htbl_entries = -1;
+
   static constexpr bool isSimpleConstant64(jlong value) {
     // Probably always true, even if a temp register is required.
     return true;
@@ -161,5 +164,7 @@
     return VM_Version::has_fcfids();
   }
 
+  // Implements a variant of EncodeISOArrayNode that encode ASCII only
+  static const bool supports_encode_ascii_array = false;
 
 #endif // CPU_PPC_MATCHER_PPC_HPP