[dilithium] Incluide AARCH64 implementation

src/sign/dilithium/dilithium2/aarch64/LICENSE

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.

src/sign/dilithium/dilithium2/aarch64/Makefile

@@ -0,0 +1,23 @@
+# This Makefile can be used with GNU Make or BSD Make
+
+LIB=libdilithium2_aarch64.a
+HEADERS=api.h fips202x2.h macros_common.inc macros.inc NTT_params.h ntt.h packing.h params.h poly.h polyvec.h reduce.h rounding.h sign.h symmetric.h
+OBJECTS=fips202x2.o ntt.o packing.o poly.o polyvec.o reduce.o rounding.o sign.o symmetric-shake.o __asm_iNTT.o __asm_NTT.o __asm_poly.o
+
+
+CFLAGS=-O3 -Wall -Wextra -Wpedantic -Werror -Wmissing-prototypes -Wredundant-decls -std=c99 -I../../../common $(EXTRAFLAGS) -g
+
+all: $(LIB)
+
+%.o: %.c $(HEADERS)
+	$(CC) $(CFLAGS) -c -o $@ $<
+
+%.o: %.S $(HEADERS)
+	$(CC) $(CFLAGS) -c -o $@ $<
+
+$(LIB): $(OBJECTS) $(HEADERS)
+	$(AR) -r $@ $(OBJECTS)
+
+clean:
+	$(RM) $(OBJECTS)
+	$(RM) $(LIB)

src/sign/dilithium/dilithium2/aarch64/NTT_params.h

@@ -0,0 +1,56 @@
+#ifndef NTT_PARAMS_H
+#define NTT_PARAMS_H
+
+#define ARRAY_N 256
+
+#define NTT_N 256
+#define LOGNTT_N 8
+
+// root of unity: 1753
+
+
+// Q1
+#define Q1 8380417
+// omegaQ1 = 1753 mod Q1
+#define omegaQ1 1753
+// invomegaQ1 = omegaQ^{-1} mod Q1
+#define invomegaQ1 731434
+// R = 2^32 below
+// RmodQ1 = 2^32 mod^{+-} Q1
+#define RmodQ1 (-4186625)
+// Q1prime = Q1^{-1} mod^{+-} 2^32
+#define Q1prime 58728449
+// invNQ1 = NTT_N^{-1} mod Q1
+#define invNQ1 8347681
+
+// invNQ1R2modQ1 = -NTT_N^{-1} 2^32 2^32 mod^{+-} Q1 below
+#define invNQ1R2modQ1 (-41978)
+// invNQ1R2modQ1_prime = invNQ1R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime 8395782
+// invNQ1R2modQ1_prime_half = (invNQ1R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime_half 4197891
+// invNQ1R2modQ1_doubleprime = (invNQ1R2modQ1_prime Q1 - invNQ1R2modQ1) / 2^32
+#define invNQ1R2modQ1_doubleprime 16382
+
+// invNQ1_final_R2modQ1 = -invNQ1R2modQ1 invomegaQ1^{128} mod q
+#define invNQ1_final_R2modQ1 4404704
+// invNQ1_final_R2modQ1_prime = invNQ1_final_R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime (-151046688)
+// invNQ1_final_R2modQ1_prime_half = (invNQ1_final_R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime_half (-75523344)
+// invNQ1_final_R2modQ1_doubleprime = (invNQ1_final_R2modQ1_prime Q1 - invNQ1_final_R2modQ1) / 2^32
+#define invNQ1_final_R2modQ1_doubleprime (-294725)
+
+// RmodQ1_prime = -(RmodQ1 + Q1) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime 512
+// RmodQ1_prime_half = ( -(RmodQ1 + Q1) / 2) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime_half 256
+// RmodQ1_doubleprime = (RmodQ1_prime Q1 - RmodQ1_prime ) / 2^32
+#define RmodQ1_doubleprime 1
+
+#endif
+
+
+
+
+

src/sign/dilithium/dilithium2/aarch64/__asm_NTT.S

@@ -0,0 +1,274 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_top
+.global _PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_top:
+_PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    src1      .req x1
+    src2      .req x2
+    src3      .req x3
+    src4      .req x4
+    src5      .req x5
+    src6      .req x6
+    src7      .req x7
+    src8      .req x8
+    src9      .req x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v1.4S}, [ src1]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v7.4S}, [ src7]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v11.4S}, [src11]
+    ld1 {v13.4S}, [src13]
+    ld1 {v15.4S}, [src15]
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v8.4S}, [ src8]
+    ld1 {v10.4S}, [src10]
+    ld1 {v12.4S}, [src12]
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    mov counter, #3
+    _ntt_top_loop:
+
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_top_loop
+
+    st1 { v1.4S}, [ src1], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v7.4S}, [ src7], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v15.4S}, [src15], #16
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v8.4S}, [ src8], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v14.4S}, [src14], #16
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_bot
+.global _PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_bot:
+_PQCLEAN_DILITHIUM2_AARCH64_asm_ntt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _ntt_bot_loop:
+
+    ld1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld1 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+
+    dq_butterfly_top  v0,  v1,  v2,  v3, v12, v13, v4,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v20, 2, 3, v20, 2, 3,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_bot v16, v18, v17, v19, v28, v29, v4, v21, 0, 1, v21, 2, 3
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_vec_top  v0,  v1,  v2,  v3, v12, v13, v4,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v22, v23, v22, v23,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_bot v16, v18, v17, v19, v28, v29, v4, v24, v25, v26, v27
+
+    st4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st4 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_bot_loop
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+
+
+
+
+
+
+
+
+

src/sign/dilithium/dilithium2/aarch64/__asm_iNTT.S

@@ -0,0 +1,509 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_top
+.global _PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_top:
+_PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    Qhalf     .req w21
+    nQhalf    .req w22
+    invNR2ph  .req w24
+    invNR2dp  .req w25
+    invNWR2ph .req w26
+    invNWR2dp .req w27
+    src0      .req  x0
+    src1      .req  x1
+    src2      .req  x2
+    src3      .req  x3
+    src4      .req  x4
+    src5      .req  x5
+    src6      .req  x6
+    src7      .req  x7
+    src8      .req  x8
+    src9      .req  x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2, #0]
+    lsr Qhalf, Q, #1
+    neg nQhalf, Qhalf
+
+    ldr invNR2ph,  [x2, #16]
+    ldr invNR2dp,  [x2, #20]
+    ldr invNWR2ph, [x2, #24]
+    ldr invNWR2dp, [x2, #28]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v1.4S}, [ src1]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v7.4S}, [ src7]
+
+    ld1 { v8.4S}, [ src8]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v10.4S}, [src10]
+    ld1 {v11.4S}, [src11]
+    ld1 {v12.4S}, [src12]
+    ld1 {v13.4S}, [src13]
+    ld1 {v14.4S}, [src14]
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    mov counter, #3
+    _intt_top_loop:
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    sub counter, counter, #1
+    cbnz counter, _intt_top_loop
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v1.4S}, [ src1], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v7.4S}, [ src7], #16
+
+    st1 { v8.4S}, [ src8], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v14.4S}, [src14], #16
+    st1 {v15.4S}, [src15], #16
+
+    .unreq    Q
+    .unreq    Qhalf
+    .unreq    nQhalf
+    .unreq    invNR2ph
+    .unreq    invNR2dp
+    .unreq    invNWR2ph
+    .unreq    invNWR2dp
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_bot
+.global _PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_bot:
+_PQCLEAN_DILITHIUM2_AARCH64_asm_intt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    RphRdp    .req x21
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+    ldr RphRdp, [x2, #8]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512    
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _intt_bot_loop:
+
+    ld4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld4 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+    mov v20.D[0], RphRdp
+
+    dq_butterfly_vec_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v24, v25, v26, v27,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_top v16, v17, v28, v29, v18, v19,  v4, v22, v23, v22, v23
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v21, 0, 1, v21, 2, 3,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_top v16, v17, v28, v29, v18, v19,  v4, v20, 2, 3, v20, 2, 3
+
+    srshr v14.4S,  v0.4S, #23
+    srshr v15.4S,  v1.4S, #23
+    srshr v30.4S, v16.4S, #23
+    srshr v31.4S, v17.4S, #23
+
+    mls    v0.4S, v14.4S, v4.S[0]
+    mls    v1.4S, v15.4S, v4.S[0]
+    mls   v16.4S, v30.4S, v4.S[0]
+    mls   v17.4S, v31.4S, v4.S[0]
+
+    st1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st1 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _intt_bot_loop
+
+    .unreq    Q
+    .unreq    RphRdp
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+

src/sign/dilithium/dilithium2/aarch64/api.h

@@ -0,0 +1,32 @@
+#ifndef PQCLEAN_DILITHIUM2_AARCH64_API_H
+#define PQCLEAN_DILITHIUM2_AARCH64_API_H
+
+
+#include <stddef.h>
+#include <stdint.h>
+
+#define PQCLEAN_DILITHIUM2_AARCH64_CRYPTO_PUBLICKEYBYTES 1312
+#define PQCLEAN_DILITHIUM2_AARCH64_CRYPTO_SECRETKEYBYTES 2528
+#define PQCLEAN_DILITHIUM2_AARCH64_CRYPTO_BYTES 2420
+#define PQCLEAN_DILITHIUM2_AARCH64_CRYPTO_ALGNAME "Dilithium2"
+
+int PQCLEAN_DILITHIUM2_AARCH64_crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+int PQCLEAN_DILITHIUM2_AARCH64_crypto_sign_signature(
+    uint8_t *sig, size_t *siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM2_AARCH64_crypto_sign_verify(
+    const uint8_t *sig, size_t siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *pk);
+
+int PQCLEAN_DILITHIUM2_AARCH64_crypto_sign(
+    uint8_t *sm, size_t *smlen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM2_AARCH64_crypto_sign_open(
+    uint8_t *m, size_t *mlen,
+    const uint8_t *sm, size_t smlen, const uint8_t *pk);
+
+
+#endif

src/sign/dilithium/dilithium2/aarch64/fips202x2.c

@@ -0,0 +1,646 @@
+#include "fips202x2.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+
+#define NROUNDS 24
+
+// Define NEON operation
+// c = load(ptr)
+#define vload(ptr) vld1q_u64(ptr);
+// ptr <= c;
+#define vstore(ptr, c) vst1q_u64(ptr, c);
+// c = a ^ b
+#define vxor(c, a, b) c = veorq_u64(a, b);
+// Rotate by n bit ((a << offset) ^ (a >> (64-offset)))
+#define vROL(out, a, offset)    \
+    (out) = vshlq_n_u64(a, offset); \
+    (out) = vsriq_n_u64(out, a, 64 - (offset));
+// Xor chain: out = a ^ b ^ c ^ d ^ e
+#define vXOR4(out, a, b, c, d, e) \
+    (out) = veorq_u64(a, b);          \
+    (out) = veorq_u64(out, c);        \
+    (out) = veorq_u64(out, d);        \
+    (out) = veorq_u64(out, e);
+// Not And c = ~a & b
+// #define vbic(c, a, b) c = vbicq_u64(b, a);
+// Xor Not And: out = a ^ ( (~b) & c)
+#define vXNA(out, a, b, c) \
+    (out) = vbicq_u64(c, b);   \
+    (out) = veorq_u64(out, a);
+// Rotate by 1 bit, then XOR: a ^ ROL(b): SHA1 instruction, not support
+#define vrxor(c, a, b) c = vrax1q_u64(a, b);
+// End Define
+
+/* Keccak round constants */
+static const uint64_t neon_KeccakF_RoundConstants[NROUNDS] = {
+    (uint64_t)0x0000000000000001ULL,
+    (uint64_t)0x0000000000008082ULL,
+    (uint64_t)0x800000000000808aULL,
+    (uint64_t)0x8000000080008000ULL,
+    (uint64_t)0x000000000000808bULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008009ULL,
+    (uint64_t)0x000000000000008aULL,
+    (uint64_t)0x0000000000000088ULL,
+    (uint64_t)0x0000000080008009ULL,
+    (uint64_t)0x000000008000000aULL,
+    (uint64_t)0x000000008000808bULL,
+    (uint64_t)0x800000000000008bULL,
+    (uint64_t)0x8000000000008089ULL,
+    (uint64_t)0x8000000000008003ULL,
+    (uint64_t)0x8000000000008002ULL,
+    (uint64_t)0x8000000000000080ULL,
+    (uint64_t)0x000000000000800aULL,
+    (uint64_t)0x800000008000000aULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008080ULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008008ULL
+};
+
+/*************************************************
+* Name:        KeccakF1600_StatePermutex2
+*
+* Description: The Keccak F1600 Permutation
+*
+* Arguments:   - uint64_t *state: pointer to input/output Keccak state
+**************************************************/
+static inline
+void KeccakF1600_StatePermutex2(v128 state[25]) {
+    v128 Aba, Abe, Abi, Abo, Abu;
+    v128 Aga, Age, Agi, Ago, Agu;
+    v128 Aka, Ake, Aki, Ako, Aku;
+    v128 Ama, Ame, Ami, Amo, Amu;
+    v128 Asa, Ase, Asi, Aso, Asu;
+    v128 BCa, BCe, BCi, BCo, BCu; // tmp
+    v128 Da, De, Di, Do, Du;      // D
+    v128 Eba, Ebe, Ebi, Ebo, Ebu;
+    v128 Ega, Ege, Egi, Ego, Egu;
+    v128 Eka, Eke, Eki, Eko, Eku;
+    v128 Ema, Eme, Emi, Emo, Emu;
+    v128 Esa, Ese, Esi, Eso, Esu;
+
+    //copyFromState(A, state)
+    Aba = state[0];
+    Abe = state[1];
+    Abi = state[2];
+    Abo = state[3];
+    Abu = state[4];
+    Aga = state[5];
+    Age = state[6];
+    Agi = state[7];
+    Ago = state[8];
+    Agu = state[9];
+    Aka = state[10];
+    Ake = state[11];
+    Aki = state[12];
+    Ako = state[13];
+    Aku = state[14];
+    Ama = state[15];
+    Ame = state[16];
+    Ami = state[17];
+    Amo = state[18];
+    Amu = state[19];
+    Asa = state[20];
+    Ase = state[21];
+    Asi = state[22];
+    Aso = state[23];
+    Asu = state[24];
+
+    for (int round = 0; round < NROUNDS; round += 2) {
+        //    prepareTheta
+        vXOR4(BCa, Aba, Aga, Aka, Ama, Asa);
+        vXOR4(BCe, Abe, Age, Ake, Ame, Ase);
+        vXOR4(BCi, Abi, Agi, Aki, Ami, Asi);
+        vXOR4(BCo, Abo, Ago, Ako, Amo, Aso);
+        vXOR4(BCu, Abu, Agu, Aku, Amu, Asu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round  , A, E)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Aba, Aba, Da);
+        vxor(Age, Age, De);
+        vROL(BCe, Age, 44);
+        vxor(Aki, Aki, Di);
+        vROL(BCi, Aki, 43);
+        vxor(Amo, Amo, Do);
+        vROL(BCo, Amo, 21);
+        vxor(Asu, Asu, Du);
+        vROL(BCu, Asu, 14);
+        vXNA(Eba, Aba, BCe, BCi);
+        vxor(Eba, Eba, vdupq_n_u64(neon_KeccakF_RoundConstants[round]));
+        vXNA(Ebe, BCe, BCi, BCo);
+        vXNA(Ebi, BCi, BCo, BCu);
+        vXNA(Ebo, BCo, BCu, Aba);
+        vXNA(Ebu, BCu, Aba, BCe);
+
+        vxor(Abo, Abo, Do);
+        vROL(BCa, Abo, 28);
+        vxor(Agu, Agu, Du);
+        vROL(BCe, Agu, 20);
+        vxor(Aka, Aka, Da);
+        vROL(BCi, Aka, 3);
+        vxor(Ame, Ame, De);
+        vROL(BCo, Ame, 45);
+        vxor(Asi, Asi, Di);
+        vROL(BCu, Asi, 61);
+        vXNA(Ega, BCa, BCe, BCi);
+        vXNA(Ege, BCe, BCi, BCo);
+        vXNA(Egi, BCi, BCo, BCu);
+        vXNA(Ego, BCo, BCu, BCa);
+        vXNA(Egu, BCu, BCa, BCe);
+
+        vxor(Abe, Abe, De);
+        vROL(BCa, Abe, 1);
+        vxor(Agi, Agi, Di);
+        vROL(BCe, Agi, 6);
+        vxor(Ako, Ako, Do);
+        vROL(BCi, Ako, 25);
+        vxor(Amu, Amu, Du);
+        vROL(BCo, Amu, 8);
+        vxor(Asa, Asa, Da);
+        vROL(BCu, Asa, 18);
+        vXNA(Eka, BCa, BCe, BCi);
+        vXNA(Eke, BCe, BCi, BCo);
+        vXNA(Eki, BCi, BCo, BCu);
+        vXNA(Eko, BCo, BCu, BCa);
+        vXNA(Eku, BCu, BCa, BCe);
+
+        vxor(Abu, Abu, Du);
+        vROL(BCa, Abu, 27);
+        vxor(Aga, Aga, Da);
+        vROL(BCe, Aga, 36);
+        vxor(Ake, Ake, De);
+        vROL(BCi, Ake, 10);
+        vxor(Ami, Ami, Di);
+        vROL(BCo, Ami, 15);
+        vxor(Aso, Aso, Do);
+        vROL(BCu, Aso, 56);
+        vXNA(Ema, BCa, BCe, BCi);
+        vXNA(Eme, BCe, BCi, BCo);
+        vXNA(Emi, BCi, BCo, BCu);
+        vXNA(Emo, BCo, BCu, BCa);
+        vXNA(Emu, BCu, BCa, BCe);
+
+        vxor(Abi, Abi, Di);
+        vROL(BCa, Abi, 62);
+        vxor(Ago, Ago, Do);
+        vROL(BCe, Ago, 55);
+        vxor(Aku, Aku, Du);
+        vROL(BCi, Aku, 39);
+        vxor(Ama, Ama, Da);
+        vROL(BCo, Ama, 41);
+        vxor(Ase, Ase, De);
+        vROL(BCu, Ase, 2);
+        vXNA(Esa, BCa, BCe, BCi);
+        vXNA(Ese, BCe, BCi, BCo);
+        vXNA(Esi, BCi, BCo, BCu);
+        vXNA(Eso, BCo, BCu, BCa);
+        vXNA(Esu, BCu, BCa, BCe);
+
+        // Next Round
+
+        //    prepareTheta
+        vXOR4(BCa, Eba, Ega, Eka, Ema, Esa);
+        vXOR4(BCe, Ebe, Ege, Eke, Eme, Ese);
+        vXOR4(BCi, Ebi, Egi, Eki, Emi, Esi);
+        vXOR4(BCo, Ebo, Ego, Eko, Emo, Eso);
+        vXOR4(BCu, Ebu, Egu, Eku, Emu, Esu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round+1, E, A)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Eba, Eba, Da);
+        vxor(Ege, Ege, De);
+        vROL(BCe, Ege, 44);
+        vxor(Eki, Eki, Di);
+        vROL(BCi, Eki, 43);
+        vxor(Emo, Emo, Do);
+        vROL(BCo, Emo, 21);
+        vxor(Esu, Esu, Du);
+        vROL(BCu, Esu, 14);
+        vXNA(Aba, Eba, BCe, BCi);
+        vxor(Aba, Aba, vdupq_n_u64(neon_KeccakF_RoundConstants[round + 1]));
+        vXNA(Abe, BCe, BCi, BCo);
+        vXNA(Abi, BCi, BCo, BCu);
+        vXNA(Abo, BCo, BCu, Eba);
+        vXNA(Abu, BCu, Eba, BCe);
+
+        vxor(Ebo, Ebo, Do);
+        vROL(BCa, Ebo, 28);
+        vxor(Egu, Egu, Du);
+        vROL(BCe, Egu, 20);
+        vxor(Eka, Eka, Da);
+        vROL(BCi, Eka, 3);
+        vxor(Eme, Eme, De);
+        vROL(BCo, Eme, 45);
+        vxor(Esi, Esi, Di);
+        vROL(BCu, Esi, 61);
+        vXNA(Aga, BCa, BCe, BCi);
+        vXNA(Age, BCe, BCi, BCo);
+        vXNA(Agi, BCi, BCo, BCu);
+        vXNA(Ago, BCo, BCu, BCa);
+        vXNA(Agu, BCu, BCa, BCe);
+
+        vxor(Ebe, Ebe, De);
+        vROL(BCa, Ebe, 1);
+        vxor(Egi, Egi, Di);
+        vROL(BCe, Egi, 6);
+        vxor(Eko, Eko, Do);
+        vROL(BCi, Eko, 25);
+        vxor(Emu, Emu, Du);
+        vROL(BCo, Emu, 8);
+        vxor(Esa, Esa, Da);
+        vROL(BCu, Esa, 18);
+        vXNA(Aka, BCa, BCe, BCi);
+        vXNA(Ake, BCe, BCi, BCo);
+        vXNA(Aki, BCi, BCo, BCu);
+        vXNA(Ako, BCo, BCu, BCa);
+        vXNA(Aku, BCu, BCa, BCe);
+
+        vxor(Ebu, Ebu, Du);
+        vROL(BCa, Ebu, 27);
+        vxor(Ega, Ega, Da);
+        vROL(BCe, Ega, 36);
+        vxor(Eke, Eke, De);
+        vROL(BCi, Eke, 10);
+        vxor(Emi, Emi, Di);
+        vROL(BCo, Emi, 15);
+        vxor(Eso, Eso, Do);
+        vROL(BCu, Eso, 56);
+        vXNA(Ama, BCa, BCe, BCi);
+        vXNA(Ame, BCe, BCi, BCo);
+        vXNA(Ami, BCi, BCo, BCu);
+        vXNA(Amo, BCo, BCu, BCa);
+        vXNA(Amu, BCu, BCa, BCe);
+
+        vxor(Ebi, Ebi, Di);
+        vROL(BCa, Ebi, 62);
+        vxor(Ego, Ego, Do);
+        vROL(BCe, Ego, 55);
+        vxor(Eku, Eku, Du);
+        vROL(BCi, Eku, 39);
+        vxor(Ema, Ema, Da);
+        vROL(BCo, Ema, 41);
+        vxor(Ese, Ese, De);
+        vROL(BCu, Ese, 2);
+        vXNA(Asa, BCa, BCe, BCi);
+        vXNA(Ase, BCe, BCi, BCo);
+        vXNA(Asi, BCi, BCo, BCu);
+        vXNA(Aso, BCo, BCu, BCa);
+        vXNA(Asu, BCu, BCa, BCe);
+    }
+
+    state[0] = Aba;
+    state[1] = Abe;
+    state[2] = Abi;
+    state[3] = Abo;
+    state[4] = Abu;
+    state[5] = Aga;
+    state[6] = Age;
+    state[7] = Agi;
+    state[8] = Ago;
+    state[9] = Agu;
+    state[10] = Aka;
+    state[11] = Ake;
+    state[12] = Aki;
+    state[13] = Ako;
+    state[14] = Aku;
+    state[15] = Ama;
+    state[16] = Ame;
+    state[17] = Ami;
+    state[18] = Amo;
+    state[19] = Amu;
+    state[20] = Asa;
+    state[21] = Ase;
+    state[22] = Asi;
+    state[23] = Aso;
+    state[24] = Asu;
+}
+
+/*************************************************
+* Name:        keccakx2_absorb
+*
+* Description: Absorb step of Keccak;
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - uint64_t *s: pointer to (uninitialized) output Keccak state
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - const uint8_t *m: pointer to input to be absorbed into s
+*              - size_t mlen: length of input in bytes
+*              - uint8_t p: domain-separation byte for different
+*                           Keccak-derived functions
+**************************************************/
+static
+void keccakx2_absorb(v128 s[25],
+                     unsigned int r,
+                     const uint8_t *in0,
+                     const uint8_t *in1,
+                     size_t inlen,
+                     uint8_t p) {
+    size_t i, pos = 0;
+
+    // Declare SIMD registers
+    v128 tmp, mask;
+    uint64x1_t a, b;
+    uint64x2_t a1, b1, atmp1, btmp1;
+    uint64x2x2_t a2, b2, atmp2, btmp2;
+    // End
+
+    for (i = 0; i < 25; ++i) {
+        s[i] = vdupq_n_u64(0);
+    }
+
+    // Load in0[i] to register, then in1[i] to register, exchange them
+    while (inlen >= r) {
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2 = vld1q_u64_x2((uint64_t *)&in0[pos]);
+            b2 = vld1q_u64_x2((uint64_t *)&in1[pos]);
+            // BD = zip1(AB and CD)
+            atmp2.val[0] = vzip1q_u64(a2.val[0], b2.val[0]);
+            atmp2.val[1] = vzip1q_u64(a2.val[1], b2.val[1]);
+            // AC = zip2(AB and CD)
+            btmp2.val[0] = vzip2q_u64(a2.val[0], b2.val[0]);
+            btmp2.val[1] = vzip2q_u64(a2.val[1], b2.val[1]);
+
+            vxor(s[i + 0], s[i + 0], atmp2.val[0]);
+            vxor(s[i + 1], s[i + 1], btmp2.val[0]);
+            vxor(s[i + 2], s[i + 2], atmp2.val[1]);
+            vxor(s[i + 3], s[i + 3], btmp2.val[1]);
+
+            pos += 8 * 2 * 2;
+        }
+        // Last iteration
+        i = r / 8 - 1;
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+        pos += 8;
+
+        KeccakF1600_StatePermutex2(s);
+        inlen -= r;
+    }
+
+    i = 0;
+    while (inlen >= 16) {
+        a1 = vld1q_u64((uint64_t *)&in0[pos]);
+        b1 = vld1q_u64((uint64_t *)&in1[pos]);
+        // BD = zip1(AB and CD)
+        atmp1 = vzip1q_u64(a1, b1);
+        // AC = zip2(AB and CD)
+        btmp1 = vzip2q_u64(a1, b1);
+
+        vxor(s[i + 0], s[i + 0], atmp1);
+        vxor(s[i + 1], s[i + 1], btmp1);
+
+        i += 2;
+        pos += 8 * 2;
+        inlen -= 8 * 2;
+    }
+
+    if (inlen >= 8) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+
+        i++;
+        pos += 8;
+        inlen -= 8;
+    }
+
+    if (inlen) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        mask = vdupq_n_u64((1ULL << (8 * inlen)) - 1);
+        tmp = vandq_u64(tmp, mask);
+        vxor(s[i], s[i], tmp);
+    }
+
+    tmp = vdupq_n_u64((uint64_t)p << (8 * inlen));
+    vxor(s[i], s[i], tmp);
+
+    mask = vdupq_n_u64(1ULL << 63);
+    vxor(s[r / 8 - 1], s[r / 8 - 1], mask);
+}
+
+/*************************************************
+* Name:        keccak_squeezeblocks
+*
+* Description: Squeeze step of Keccak. Squeezes full blocks of r bytes each.
+*              Modifies the state. Can be called multiple times to keep
+*              squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out: pointer to output blocks
+*              - size_t nblocks: number of blocks to be squeezed (written to h)
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - uint64_t *s: pointer to input/output Keccak state
+**************************************************/
+static
+void keccakx2_squeezeblocks(uint8_t *out0,
+                            uint8_t *out1,
+                            size_t nblocks,
+                            unsigned int r,
+                            v128 s[25]) {
+    unsigned int i;
+
+    uint64x1_t a, b;
+    uint64x2x2_t a2, b2;
+
+    while (nblocks > 0) {
+        KeccakF1600_StatePermutex2(s);
+
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2.val[0] = vuzp1q_u64(s[i], s[i + 1]);
+            b2.val[0] = vuzp2q_u64(s[i], s[i + 1]);
+            a2.val[1] = vuzp1q_u64(s[i + 2], s[i + 3]);
+            b2.val[1] = vuzp2q_u64(s[i + 2], s[i + 3]);
+            vst1q_u64_x2((uint64_t *)out0, a2);
+            vst1q_u64_x2((uint64_t *)out1, b2);
+
+            out0 += 32;
+            out1 += 32;
+        }
+
+        i = r / 8 - 1;
+        // Last iteration
+        a = vget_low_u64(s[i]);
+        b = vget_high_u64(s[i]);
+        vst1_u64((uint64_t *)out0, a);
+        vst1_u64((uint64_t *)out1, b);
+
+        out0 += 8;
+        out1 += 8;
+
+        --nblocks;
+    }
+}
+
+/*************************************************
+* Name:        shake128x2_absorb
+*
+* Description: Absorb step of the SHAKE128 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *state: pointer to (uninitialized) output
+*                                     Keccak state
+*              - const uint8_t *in:   pointer to input to be absorbed into s
+*              - size_t inlen:        length of input in bytes
+**************************************************/
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE128_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake128_squeezeblocks
+*
+* Description: Squeeze step of SHAKE128 XOF. Squeezes full blocks of
+*              SHAKE128_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE128_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake256_absorb
+*
+* Description: Absorb step of the SHAKE256 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *s:   pointer to (uninitialized) output Keccak state
+*              - const uint8_t *in: pointer to input to be absorbed into s
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE256_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake256_squeezeblocks
+*
+* Description: Squeeze step of SHAKE256 XOF. Squeezes full blocks of
+*              SHAKE256_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE256_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake128
+*
+* Description: SHAKE128 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE128_RATE;
+    uint8_t t[2][SHAKE128_RATE];
+    keccakx2_state state;
+
+    shake128x2_absorb(&state, in0, in1, inlen);
+    shake128x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE128_RATE;
+    out1 += nblocks * SHAKE128_RATE;
+    outlen -= nblocks * SHAKE128_RATE;
+
+    if (outlen) {
+        shake128x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}
+
+/*************************************************
+* Name:        shake256
+*
+* Description: SHAKE256 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE256_RATE;
+    uint8_t t[2][SHAKE256_RATE];
+    keccakx2_state state;
+
+    shake256x2_absorb(&state, in0, in1, inlen);
+    shake256x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE256_RATE;
+    out1 += nblocks * SHAKE256_RATE;
+    outlen -= nblocks * SHAKE256_RATE;
+
+    if (outlen) {
+        shake256x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}

src/sign/dilithium/dilithium2/aarch64/fips202x2.h

@@ -0,0 +1,60 @@
+#ifndef FIPS202X2_H
+#define FIPS202X2_H
+
+#include "params.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+typedef uint64x2_t v128;
+
+#define SHAKE128_RATE 168
+#define SHAKE256_RATE 136
+#define SHA3_256_RATE 136
+#define SHA3_512_RATE 72
+
+
+typedef struct {
+    v128 s[25];
+} keccakx2_state;
+
+
+#define shake128x2_absorb DILITHIUM_NAMESPACE(shake128x2_absorb)
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake128x2_squeezeblocks DILITHIUM_NAMESPACE(shake128x2_squeezeblocks)
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake256x2_absorb DILITHIUM_NAMESPACE(shake256x2_absorb)
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake256x2_squeezeblocks DILITHIUM_NAMESPACE(shake256x2_squeezeblocks)
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake128x2 DILITHIUM_NAMESPACE(shake128x2)
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+
+#define shake256x2 DILITHIUM_NAMESPACE(shake256x2)
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+#endif

src/sign/dilithium/dilithium2/aarch64/macros.inc

@@ -0,0 +1,84 @@
+#include "macros_common.inc"
+
+.macro wrap_trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3, qS, dD
+
+    trn1 \t0\qS, \a0\qS, \a1\qS
+    trn2 \t1\qS, \a0\qS, \a1\qS
+    trn1 \t2\qS, \a2\qS, \a3\qS
+    trn2 \t3\qS, \a2\qS, \a3\qS
+
+    trn1 \a0\dD, \t0\dD, \t2\dD
+    trn2 \a2\dD, \t0\dD, \t2\dD
+    trn1 \a1\dD, \t1\dD, \t3\dD
+    trn2 \a3\dD, \t1\dD, \t3\dD
+
+.endm
+
+.macro trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3
+    wrap_trn_4x4 \a0, \a1, \a2, \a3, \t0, \t1, \t2, \t3, .4S, .2D
+.endm
+
+
+.macro dq_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+
+.macro dq_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_montgomery_mul \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_top \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_bot \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed_rev \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+
+.macro qq_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q
+    wrap_qX_montgomery \c0, \c1, \c2, \c3, \l0, \l1, \l2, \l3, \h0, \h1, \h2, \h3, \t0, \t1, \t2, \t3, \Qprime, \Q, .2S, .4S, .2D
+.endm
+
+.macro qq_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3
+    wrap_qX_sub_add \s0, \s1, \s2, \s3, \t0, \t1, \t2, \t3, \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, .4S
+.endm

src/sign/dilithium/dilithium2/aarch64/macros_common.inc

@@ -0,0 +1,423 @@
+// for ABI
+
+.macro push_all
+
+    sub sp, sp, #(16*9)
+    stp x19, x20, [sp, #16*0]
+    stp x21, x22, [sp, #16*1]
+    stp x23, x24, [sp, #16*2]
+    stp x25, x26, [sp, #16*3]
+    stp x27, x28, [sp, #16*4]
+    stp  d8,  d9, [sp, #16*5]
+    stp d10, d11, [sp, #16*6]
+    stp d12, d13, [sp, #16*7]
+    stp d14, d15, [sp, #16*8]
+
+.endm
+
+.macro pop_all
+
+    ldp x19, x20, [sp, #16*0]
+    ldp x21, x22, [sp, #16*1]
+    ldp x23, x24, [sp, #16*2]
+    ldp x25, x26, [sp, #16*3]
+    ldp x27, x28, [sp, #16*4]
+    ldp  d8,  d9, [sp, #16*5]
+    ldp d10, d11, [sp, #16*6]
+    ldp d12, d13, [sp, #16*7]
+    ldp d14, d15, [sp, #16*8]
+    add sp, sp, #(16*9)
+
+.endm
+
+// vector-scalar butterflies
+
+.macro wrap_dX_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+    sub     \b2\wX, \a2\wX, \t2\wX
+    sub     \b3\wX, \a3\wX, \t3\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+    add     \a2\wX, \a2\wX, \t2\wX
+    add     \a3\wX, \a3\wX, \t3\wX
+
+.endm
+
+.macro wrap_qX_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t4\wX, \b4\wX, \z4\nX[\h4]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t5\wX, \b5\wX, \z5\nX[\h5]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t6\wX, \b6\wX, \z6\nX[\h6]
+    sub      \b3\wX, \a3\wX, \t3\wX
+    mul      \t7\wX, \b7\wX, \z7\nX[\h7]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b4\wX, \b4\wX, \z4\nX[\l4]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b5\wX, \b5\wX, \z5\nX[\l5]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b6\wX, \b6\wX, \z6\nX[\l6]
+    add      \a3\wX, \a3\wX, \t3\wX
+    sqrdmulh \b7\wX, \b7\wX, \z7\nX[\l7]
+
+    mls      \t4\wX, \b4\wX, \mod\nX[0]
+    mls      \t5\wX, \b5\wX, \mod\nX[0]
+    mls      \t6\wX, \b6\wX, \mod\nX[0]
+    mls      \t7\wX, \b7\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b4\wX, \a4\wX, \t4\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b5\wX, \a5\wX, \t5\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b6\wX, \a6\wX, \t6\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+    sub      \b7\wX, \a7\wX, \t7\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a4\wX, \a4\wX, \t4\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a5\wX, \a5\wX, \t5\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a6\wX, \a6\wX, \t6\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+    add      \a7\wX, \a7\wX, \t7\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+// vector-vector butterflies
+
+.macro wrap_dX_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \h2\wX
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \h3\wX
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \l2\wX
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \l3\wX
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+// vector-scalar Barrett reduction
+
+.macro wrap_qX_barrett a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oX_barrett a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\nX[0]
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\nX[0]
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\nX[0]
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// vector-vector Barrett reduction
+
+.macro wrap_qo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\wX
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\wX
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// Montgomery multiplication
+
+.macro wrap_qX_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \b0\wX, \t0\wX, \z0\nX[\h0]
+    mul      \b1\wX, \t1\wX, \z1\nX[\h1]
+    mul      \b2\wX, \t2\wX, \z2\nX[\h2]
+    mul      \b3\wX, \t3\wX, \z3\nX[\h3]
+
+    sqrdmulh \t0\wX, \t0\wX, \z0\nX[\l0]
+    sqrdmulh \t1\wX, \t1\wX, \z1\nX[\l1]
+    sqrdmulh \t2\wX, \t2\wX, \z2\nX[\l2]
+    sqrdmulh \t3\wX, \t3\wX, \z3\nX[\l3]
+
+    mls      \b0\wX, \t0\wX, \mod\nX[0]
+    mls      \b1\wX, \t1\wX, \mod\nX[0]
+    mls      \b2\wX, \t2\wX, \mod\nX[0]
+    mls      \b3\wX, \t3\wX, \mod\nX[0]
+
+.endm
+
+// Montgomery reduction with long
+
+.macro wrap_qX_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q, lX, wX, dwX
+
+    uzp1 \t0\wX, \l0\wX, \h0\wX
+    uzp1 \t1\wX, \l1\wX, \h1\wX
+    uzp1 \t2\wX, \l2\wX, \h2\wX
+    uzp1 \t3\wX, \l3\wX, \h3\wX
+
+    mul \t0\wX, \t0\wX, \Qprime\wX
+    mul \t1\wX, \t1\wX, \Qprime\wX
+    mul \t2\wX, \t2\wX, \Qprime\wX
+    mul \t3\wX, \t3\wX, \Qprime\wX
+
+    smlal  \l0\dwX, \t0\lX, \Q\lX
+    smlal2 \h0\dwX, \t0\wX, \Q\wX
+    smlal  \l1\dwX, \t1\lX, \Q\lX
+    smlal2 \h1\dwX, \t1\wX, \Q\wX
+    smlal  \l2\dwX, \t2\lX, \Q\lX
+    smlal2 \h2\dwX, \t2\wX, \Q\wX
+    smlal  \l3\dwX, \t3\lX, \Q\lX
+    smlal2 \h3\dwX, \t3\wX, \Q\wX
+
+    uzp2 \c0\wX, \l0\wX, \h0\wX
+    uzp2 \c1\wX, \l1\wX, \h1\wX
+    uzp2 \c2\wX, \l2\wX, \h2\wX
+    uzp2 \c3\wX, \l3\wX, \h3\wX
+
+.endm
+
+// add_sub, sub_add
+
+.macro wrap_qX_add_sub s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s3\wX, \a3\wX, \b3\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+
+.endm
+
+.macro wrap_qX_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+    add \s3\wX, \a3\wX, \b3\wX
+
+.endm

src/sign/dilithium/dilithium2/aarch64/ntt.c

@@ -0,0 +1,35 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+#include <stdio.h>
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+
+/*************************************************
+* Name:        ntt
+*
+* Description: Forward NTT, in-place. No modular reduction is performed after
+*              additions or subtractions. Output vector is in bitreversed order.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void ntt(int32_t a[N]) {
+    NTT(a);
+}
+
+/*************************************************
+* Name:        invntt_tomont
+*
+* Description: Inverse NTT and multiplication by Montgomery factor 2^32.
+*              In-place. No modular reductions after additions or
+*              subtractions; input coefficients need to be smaller than
+*              Q in absolute value. Output coefficient are smaller than Q in
+*              absolute value.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void invntt_tomont(int32_t a[N]) {
+    iNTT(a);
+}

src/sign/dilithium/dilithium2/aarch64/packing.c

@@ -0,0 +1,261 @@
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+
+
+/*************************************************
+* Name:        pack_pk
+*
+* Description: Bit-pack public key pk = (rho, t1).
+*
+* Arguments:   - uint8_t pk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const polyveck *t1: pointer to vector t1
+**************************************************/
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const polyveck *t1) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        pk[i] = rho[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_pack(pk + i * POLYT1_PACKEDBYTES, &t1->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_pk
+*
+* Description: Unpack public key pk = (rho, t1).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const polyveck *t1: pointer to output vector t1
+*              - uint8_t pk[]: byte array containing bit-packed pk
+**************************************************/
+void unpack_pk(uint8_t rho[SEEDBYTES],
+               polyveck *t1,
+               const uint8_t pk[CRYPTO_PUBLICKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = pk[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_unpack(&t1->vec[i], pk + i * POLYT1_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sk
+*
+* Description: Bit-pack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - uint8_t sk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const uint8_t tr[]: byte array containing tr
+*              - const uint8_t key[]: byte array containing key
+*              - const polyveck *t0: pointer to vector t0
+*              - const polyvecl *s1: pointer to vector s1
+*              - const polyveck *s2: pointer to vector s2
+**************************************************/
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = rho[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = key[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = tr[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s1->vec[i]);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s2->vec[i]);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_pack(sk + i * POLYT0_PACKEDBYTES, &t0->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_sk
+*
+* Description: Unpack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const uint8_t tr[]: output byte array for tr
+*              - const uint8_t key[]: output byte array for key
+*              - const polyveck *t0: pointer to output vector t0
+*              - const polyvecl *s1: pointer to output vector s1
+*              - const polyveck *s2: pointer to output vector s2
+*              - uint8_t sk[]: byte array containing bit-packed sk
+**************************************************/
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        key[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        tr[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_unpack(&s1->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_unpack(&s2->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_unpack(&t0->vec[i], sk + i * POLYT0_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sig
+*
+* Description: Bit-pack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t sig[]: output byte array
+*              - const uint8_t *c: pointer to challenge hash length SEEDBYTES
+*              - const polyvecl *z: pointer to vector z
+*              - const polyveck *h: pointer to hint vector h
+**************************************************/
+void pack_sig(uint8_t sig[CRYPTO_BYTES],
+              const uint8_t c[SEEDBYTES],
+              const polyvecl *z,
+              const polyveck *h) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sig[i] = c[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_pack(sig + i * POLYZ_PACKEDBYTES, &z->vec[i]);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Encode h */
+    for (i = 0; i < OMEGA + K; ++i) {
+        sig[i] = 0;
+    }
+
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            if (h->vec[i].coeffs[j] != 0) {
+                sig[k++] = (uint8_t) j;
+            }
+        }
+
+        sig[OMEGA + i] = (uint8_t) k;
+    }
+}
+
+/*************************************************
+* Name:        unpack_sig
+*
+* Description: Unpack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t *c: pointer to output challenge hash
+*              - polyvecl *z: pointer to output vector z
+*              - polyveck *h: pointer to output hint vector h
+*              - const uint8_t sig[]: byte array containing
+*                bit-packed signature
+*
+* Returns 1 in case of malformed signature; otherwise 0.
+**************************************************/
+int unpack_sig(uint8_t c[SEEDBYTES],
+               polyvecl *z,
+               polyveck *h,
+               const uint8_t sig[CRYPTO_BYTES]) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        c[i] = sig[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_unpack(&z->vec[i], sig + i * POLYZ_PACKEDBYTES);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Decode h */
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            h->vec[i].coeffs[j] = 0;
+        }
+
+        if (sig[OMEGA + i] < k || sig[OMEGA + i] > OMEGA) {
+            return 1;
+        }
+
+        for (j = k; j < sig[OMEGA + i]; ++j) {
+            /* Coefficients are ordered for strong unforgeability */
+            if (j > k && sig[j] <= sig[j - 1]) {
+                return 1;
+            }
+            h->vec[i].coeffs[sig[j]] = 1;
+        }
+
+        k = sig[OMEGA + i];
+    }
+
+    /* Extra indices are zero for strong unforgeability */
+    for (j = k; j < OMEGA; ++j) {
+        if (sig[j]) {
+            return 1;
+        }
+    }
+
+    return 0;
+}

src/sign/dilithium/dilithium2/aarch64/packing.h

@@ -0,0 +1,37 @@
+#ifndef PACKING_H
+#define PACKING_H
+#include "params.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#define pack_pk DILITHIUM_NAMESPACE(pack_pk)
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES], const uint8_t rho[SEEDBYTES], const polyveck *t1);
+
+#define pack_sk DILITHIUM_NAMESPACE(pack_sk)
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2);
+
+#define pack_sig DILITHIUM_NAMESPACE(pack_sig)
+void pack_sig(uint8_t sig[CRYPTO_BYTES], const uint8_t c[SEEDBYTES], const polyvecl *z, const polyveck *h);
+
+#define unpack_pk DILITHIUM_NAMESPACE(unpack_pk)
+void unpack_pk(uint8_t rho[SEEDBYTES], polyveck *t1, const uint8_t pk[CRYPTO_PUBLICKEYBYTES]);
+
+#define unpack_sk DILITHIUM_NAMESPACE(unpack_sk)
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]);
+
+#define unpack_sig DILITHIUM_NAMESPACE(unpack_sig)
+int unpack_sig(uint8_t c[SEEDBYTES], polyvecl *z, polyveck *h, const uint8_t sig[CRYPTO_BYTES]);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/params.h

@@ -0,0 +1,51 @@
+#ifndef PARAMS_H
+#define PARAMS_H
+
+#define DILITHIUM_MODE 2
+//#define DILITHIUM_MODE 3
+//#define DILITHIUM_MODE 5
+
+#define CRYPTO_NAMESPACETOP PQCLEAN_DILITHIUM2_AARCH64_crypto_sign
+#define CRYPTO_NAMESPACE(s) PQCLEAN_DILITHIUM2_AARCH64_##s
+#define DILITHIUM_NAMESPACETOP CRYPTO_NAMESPACETOP
+#define DILITHIUM_NAMESPACE(s) CRYPTO_NAMESPACE(s)
+
+
+#define SEEDBYTES 32
+#define CRHBYTES 64
+#define N 256
+#define DILITHIUM_Q 8380417
+#define D 13
+#define ROOT_OF_UNITY 1753
+
+
+#define K 4
+#define L 4
+#define ETA 2
+#define TAU 39
+#define BETA 78
+#define GAMMA1 (1 << 17)
+#define GAMMA2 ((DILITHIUM_Q-1)/88)
+#define OMEGA 80
+#define CRYPTO_ALGNAME "Dilithium2"
+
+
+#define POLYT1_PACKEDBYTES  320
+#define POLYT0_PACKEDBYTES  416
+#define POLYVECH_PACKEDBYTES (OMEGA + K)
+
+
+#define POLYZ_PACKEDBYTES   576
+
+#define POLYW1_PACKEDBYTES  192
+
+#define POLYETA_PACKEDBYTES  96
+
+#define CRYPTO_PUBLICKEYBYTES (SEEDBYTES + K*POLYT1_PACKEDBYTES)
+#define CRYPTO_SECRETKEYBYTES (3*SEEDBYTES \
+                               + L*POLYETA_PACKEDBYTES \
+                               + K*POLYETA_PACKEDBYTES \
+                               + K*POLYT0_PACKEDBYTES)
+#define CRYPTO_BYTES (SEEDBYTES + L*POLYZ_PACKEDBYTES + POLYVECH_PACKEDBYTES)
+
+#endif

src/sign/dilithium/dilithium2/aarch64/poly.c

@@ -0,0 +1,940 @@
+#include "params.h"
+#include "poly.h"
+#include "reduce.h"
+#include "rounding.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+#include "fips202x2.h"
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+static const int32_t montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+#define DBENCH_START()
+#define DBENCH_STOP(t)
+
+/*************************************************
+* Name:        poly_reduce
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              representative in [-6283009,6283007].
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_poly_reduce(int32_t *, const int32_t *);
+void poly_reduce(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_poly_reduce(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_caddq
+*
+* Description: For all coefficients of in/out polynomial add Q if
+*              coefficient is negative.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_poly_caddq(int32_t *, const int32_t *);
+void poly_caddq(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_poly_caddq(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_freeze
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              standard representatives.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_poly_freeze(int32_t *, const int32_t *);
+void poly_freeze(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_poly_freeze(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_add
+*
+* Description: Add polynomials. No modular reduction is performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first summand
+*              - const poly *b: pointer to second summand
+**************************************************/
+void poly_add(poly *c, const poly *a, const poly *b)  {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] + b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_sub
+*
+* Description: Subtract polynomials. No modular reduction is
+*              performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial to be
+*                               subtraced from first input polynomial
+**************************************************/
+void poly_sub(poly *c, const poly *a, const poly *b) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] - b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_shiftl
+*
+* Description: Multiply polynomial by 2^D without modular reduction. Assumes
+*              input coefficients to be less than 2^{31-D} in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_shiftl(poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a->coeffs[i] <<= D;
+    }
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_ntt
+*
+* Description: Inplace forward NTT. Coefficients can grow by
+*              8*Q in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_ntt(poly *a) {
+    DBENCH_START();
+
+    ntt(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_invntt_tomont
+*
+* Description: Inplace inverse NTT and multiplication by 2^{32}.
+*              Input coefficients need to be less than Q in absolute
+*              value and output coefficients are again bounded by Q.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_invntt_tomont(poly *a) {
+    DBENCH_START();
+
+    invntt_tomont(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_pointwise_montgomery
+*
+* Description: Pointwise multiplication of polynomials in NTT domain
+*              representation and multiplication of resulting polynomial
+*              by 2^{-32}.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_poly_pointwise_montgomery(int32_t *des, const int32_t *src1, const int32_t *src2, const int32_t *table);
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_poly_pointwise_montgomery(c->coeffs, a->coeffs, b->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_power2round
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute c0, c1 such that c mod Q = c1*2^D + c0
+*              with -2^{D-1} < c0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_poly_power2round(int32_t *, int32_t *, const int32_t *);
+void poly_power2round(poly *a1, poly *a0, const poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_poly_power2round(a1->coeffs, a0->coeffs, a->coeffs);
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_decompose
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute high and low bits c0, c1 such c mod Q = c1*ALPHA + c0
+*              with -ALPHA/2 < c0 <= ALPHA/2 except c1 = (Q-1)/ALPHA where we
+*              set c1 = 0 and -ALPHA/2 <= c0 = c mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void poly_decompose(poly *a1, poly *a0, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a1->coeffs[i] = decompose(&a0->coeffs[i], a->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_make_hint
+*
+* Description: Compute hint polynomial. The coefficients of which indicate
+*              whether the low bits of the corresponding coefficient of
+*              the input polynomial overflow into the high bits.
+*
+* Arguments:   - poly *h: pointer to output hint polynomial
+*              - const poly *a0: pointer to low part of input polynomial
+*              - const poly *a1: pointer to high part of input polynomial
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1) {
+    unsigned int i, s = 0;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        h->coeffs[i] = make_hint(a0->coeffs[i], a1->coeffs[i]);
+        s += h->coeffs[i];
+    }
+
+    DBENCH_STOP(*tround);
+    return s;
+}
+
+/*************************************************
+* Name:        poly_use_hint
+*
+* Description: Use hint polynomial to correct the high bits of a polynomial.
+*
+* Arguments:   - poly *b: pointer to output polynomial with corrected high bits
+*              - const poly *a: pointer to input polynomial
+*              - const poly *h: pointer to input hint polynomial
+**************************************************/
+void poly_use_hint(poly *b, const poly *a, const poly *h) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        b->coeffs[i] = use_hint(a->coeffs[i], h->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_chknorm
+*
+* Description: Check infinity norm of polynomial against given bound.
+*              Assumes input coefficients were reduced by reduce32().
+*
+* Arguments:   - const poly *a: pointer to polynomial
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm is strictly smaller than B <= (Q-1)/8 and 1 otherwise.
+**************************************************/
+int poly_chknorm(const poly *a, int32_t B) {
+    unsigned int i;
+    int32_t t;
+    DBENCH_START();
+
+    if (B > (DILITHIUM_Q - 1) / 8) {
+        return 1;
+    }
+
+    /* It is ok to leak which coefficient violates the bound since
+       the probability for each coefficient is independent of secret
+       data but we must not leak the sign of the centralized representative. */
+    for (i = 0; i < N; ++i) {
+        /* Absolute value */
+        t = a->coeffs[i] >> 31;
+        t = a->coeffs[i] - (t & 2 * a->coeffs[i]);
+
+        if (t >= B) {
+            DBENCH_STOP(*tsample);
+            return 1;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return 0;
+}
+
+/*************************************************
+* Name:        rej_uniform
+*
+* Description: Sample uniformly random coefficients in [0, Q-1] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_uniform(int32_t *a,
+                                unsigned int len,
+                                const uint8_t *buf,
+                                unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos + 3 <= buflen) {
+        t  = buf[pos++];
+        t |= (uint32_t)buf[pos++] << 8;
+        t |= (uint32_t)buf[pos++] << 16;
+        t &= 0x7FFFFF;
+
+        if (t < DILITHIUM_Q) {
+            a[ctr++] = t;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [0,Q-1] by performing rejection sampling on the
+*              output stream of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length SEEDBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+
+#define POLY_UNIFORM_NBLOCKS ((768 + STREAM128_BLOCKBYTES - 1)/STREAM128_BLOCKBYTES)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce) {
+    unsigned int i, ctr, off;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    stream128_state state;
+
+    stream128_init(&state, seed, nonce);
+    stream128_squeezeblocks(buf, POLY_UNIFORM_NBLOCKS, &state);
+
+    ctr = rej_uniform(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        off = buflen % 3;
+        for (i = 0; i < off; ++i) {
+            buf[i] = buf[buflen - off + i];
+        }
+
+        stream128_squeezeblocks(buf + off, 1, &state);
+        buflen = STREAM128_BLOCKBYTES + off;
+        ctr += rej_uniform(a->coeffs + ctr, N - ctr, buf, buflen);
+    }
+    stream128_release(&state);
+}
+
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf0[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    uint8_t buf1[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+
+    keccakx2_state statex2;
+    dilithium_shake128x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake128x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_NBLOCKS, &statex2);
+
+    ctr0 = rej_uniform(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_uniform(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake128x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_uniform(a0->coeffs + ctr0, N - ctr0, buf0, buflen);
+        ctr1 += rej_uniform(a1->coeffs + ctr1, N - ctr1, buf1, buflen);
+    }
+
+
+}
+
+/*************************************************
+* Name:        rej_eta
+*
+* Description: Sample uniformly random coefficients in [-ETA, ETA] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_eta(int32_t *a,
+                            unsigned int len,
+                            const uint8_t *buf,
+                            unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t0, t1;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos < buflen) {
+        t0 = buf[pos] & 0x0F;
+        t1 = buf[pos++] >> 4;
+
+
+        if (t0 < 15) {
+            t0 = t0 - (205 * t0 >> 10) * 5;
+            a[ctr++] = 2 - t0;
+        }
+        if (t1 < 15 && ctr < len) {
+            t1 = t1 - (205 * t1 >> 10) * 5;
+            a[ctr++] = 2 - t1;
+        }
+
+
+
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform_eta
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-ETA,ETA] by performing rejection sampling on the
+*              output stream from SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+#define POLY_UNIFORM_ETA_NBLOCKS ((136 + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce) {
+    unsigned int ctr;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_ETA_NBLOCKS, &state);
+
+    ctr = rej_eta(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        stream256_squeezeblocks(buf, 1, &state);
+        ctr += rej_eta(a->coeffs + ctr, N - ctr, buf, STREAM256_BLOCKBYTES);
+    }
+    stream256_release(&state);
+}
+
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+
+    uint8_t buf0[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_ETA_NBLOCKS, &statex2);
+
+    ctr0 = rej_eta(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_eta(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake256x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_eta(a0->coeffs + ctr0, N - ctr0, buf0, STREAM256_BLOCKBYTES);
+        ctr1 += rej_eta(a1->coeffs + ctr1, N - ctr1, buf1, STREAM256_BLOCKBYTES);
+    }
+}
+
+/*************************************************
+* Name:        poly_uniform_gamma1m1
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-(GAMMA1 - 1), GAMMA1] by unpacking output stream
+*              of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 16-bit nonce
+**************************************************/
+#define POLY_UNIFORM_GAMMA1_NBLOCKS ((POLYZ_PACKEDBYTES + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce) {
+    uint8_t buf[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_GAMMA1_NBLOCKS, &state);
+    stream256_release(&state);
+    polyz_unpack(a, buf);
+}
+
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1) {
+
+    uint8_t buf0[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_GAMMA1_NBLOCKS, &statex2);
+
+    polyz_unpack(a0, buf0);
+    polyz_unpack(a1, buf1);
+
+}
+
+/*************************************************
+* Name:        challenge
+*
+* Description: Implementation of H. Samples polynomial with TAU nonzero
+*              coefficients in {-1,1} using the output stream of
+*              SHAKE256(seed).
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const uint8_t mu[]: byte array containing seed of length SEEDBYTES
+**************************************************/
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]) {
+    unsigned int i, b, pos;
+    uint64_t signs;
+    uint8_t buf[SHAKE256_RATE];
+    shake256incctx state;
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, seed, SEEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(buf, sizeof buf, &state);
+
+    signs = 0;
+    for (i = 0; i < 8; ++i) {
+        signs |= (uint64_t)buf[i] << 8 * i;
+    }
+    pos = 8;
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = 0;
+    }
+    for (i = N - TAU; i < N; ++i) {
+        do {
+            if (pos >= SHAKE256_RATE) {
+                shake256_inc_squeeze(buf, sizeof buf, &state);
+                pos = 0;
+            }
+
+            b = buf[pos++];
+        } while (b > i);
+
+        c->coeffs[i] = c->coeffs[b];
+        c->coeffs[b] = 1 - 2 * (signs & 1);
+        signs >>= 1;
+    }
+    shake256_inc_ctx_release(&state);
+}
+
+/*************************************************
+* Name:        polyeta_pack
+*
+* Description: Bit-pack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYETA_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyeta_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint8_t t[8];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 8; ++i) {
+        t[0] = ETA - a->coeffs[8 * i + 0];
+        t[1] = ETA - a->coeffs[8 * i + 1];
+        t[2] = ETA - a->coeffs[8 * i + 2];
+        t[3] = ETA - a->coeffs[8 * i + 3];
+        t[4] = ETA - a->coeffs[8 * i + 4];
+        t[5] = ETA - a->coeffs[8 * i + 5];
+        t[6] = ETA - a->coeffs[8 * i + 6];
+        t[7] = ETA - a->coeffs[8 * i + 7];
+
+        r[3 * i + 0]  = (t[0] >> 0) | (t[1] << 3) | (t[2] << 6);
+        r[3 * i + 1]  = (t[2] >> 2) | (t[3] << 1) | (t[4] << 4) | (t[5] << 7);
+        r[3 * i + 2]  = (t[5] >> 1) | (t[6] << 2) | (t[7] << 5);
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyeta_unpack
+*
+* Description: Unpack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyeta_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 8; ++i) {
+        r->coeffs[8 * i + 0] =  (a[3 * i + 0] >> 0) & 7;
+        r->coeffs[8 * i + 1] =  (a[3 * i + 0] >> 3) & 7;
+        r->coeffs[8 * i + 2] = ((a[3 * i + 0] >> 6) | (a[3 * i + 1] << 2)) & 7;
+        r->coeffs[8 * i + 3] =  (a[3 * i + 1] >> 1) & 7;
+        r->coeffs[8 * i + 4] =  (a[3 * i + 1] >> 4) & 7;
+        r->coeffs[8 * i + 5] = ((a[3 * i + 1] >> 7) | (a[3 * i + 2] << 1)) & 7;
+        r->coeffs[8 * i + 6] =  (a[3 * i + 2] >> 2) & 7;
+        r->coeffs[8 * i + 7] =  (a[3 * i + 2] >> 5) & 7;
+
+        r->coeffs[8 * i + 0] = ETA - r->coeffs[8 * i + 0];
+        r->coeffs[8 * i + 1] = ETA - r->coeffs[8 * i + 1];
+        r->coeffs[8 * i + 2] = ETA - r->coeffs[8 * i + 2];
+        r->coeffs[8 * i + 3] = ETA - r->coeffs[8 * i + 3];
+        r->coeffs[8 * i + 4] = ETA - r->coeffs[8 * i + 4];
+        r->coeffs[8 * i + 5] = ETA - r->coeffs[8 * i + 5];
+        r->coeffs[8 * i + 6] = ETA - r->coeffs[8 * i + 6];
+        r->coeffs[8 * i + 7] = ETA - r->coeffs[8 * i + 7];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_pack
+*
+* Description: Bit-pack polynomial t1 with coefficients fitting in 10 bits.
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 4; ++i) {
+        r[5 * i + 0] = (uint8_t) (a->coeffs[4 * i + 0] >> 0);
+        r[5 * i + 1] = (uint8_t) ((a->coeffs[4 * i + 0] >> 8) | (a->coeffs[4 * i + 1] << 2));
+        r[5 * i + 2] = (uint8_t) ((a->coeffs[4 * i + 1] >> 6) | (a->coeffs[4 * i + 2] << 4));
+        r[5 * i + 3] = (uint8_t) ((a->coeffs[4 * i + 2] >> 4) | (a->coeffs[4 * i + 3] << 6));
+        r[5 * i + 4] = (uint8_t) (a->coeffs[4 * i + 3] >> 2);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_unpack
+*
+* Description: Unpack polynomial t1 with 10-bit coefficients.
+*              Output coefficients are standard representatives.
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_10_to_32(int32_t *, const uint8_t *);
+void polyt1_unpack(poly *r, const uint8_t *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM2_AARCH64_asm_10_to_32(r->coeffs, a);
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_pack
+*
+* Description: Bit-pack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT0_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt0_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[8];
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        t[0] = (1 << (D - 1)) - a->coeffs[8 * i + 0];
+        t[1] = (1 << (D - 1)) - a->coeffs[8 * i + 1];
+        t[2] = (1 << (D - 1)) - a->coeffs[8 * i + 2];
+        t[3] = (1 << (D - 1)) - a->coeffs[8 * i + 3];
+        t[4] = (1 << (D - 1)) - a->coeffs[8 * i + 4];
+        t[5] = (1 << (D - 1)) - a->coeffs[8 * i + 5];
+        t[6] = (1 << (D - 1)) - a->coeffs[8 * i + 6];
+        t[7] = (1 << (D - 1)) - a->coeffs[8 * i + 7];
+
+        r[13 * i + 0]  =  (uint8_t) t[0];
+        r[13 * i + 1]  =  (uint8_t) (t[0] >>  8);
+        r[13 * i + 1] |=  (uint8_t) (t[1] <<  5);
+        r[13 * i + 2]  =  (uint8_t) (t[1] >>  3);
+        r[13 * i + 3]  =  (uint8_t) (t[1] >> 11);
+        r[13 * i + 3] |=  (uint8_t) (t[2] <<  2);
+        r[13 * i + 4]  =  (uint8_t) (t[2] >>  6);
+        r[13 * i + 4] |=  (uint8_t) (t[3] <<  7);
+        r[13 * i + 5]  =  (uint8_t) (t[3] >>  1);
+        r[13 * i + 6]  =  (uint8_t) (t[3] >>  9);
+        r[13 * i + 6] |=  (uint8_t) (t[4] <<  4);
+        r[13 * i + 7]  =  (uint8_t) (t[4] >>  4);
+        r[13 * i + 8]  =  (uint8_t) (t[4] >> 12);
+        r[13 * i + 8] |=  (uint8_t) (t[5] <<  1);
+        r[13 * i + 9]  =  (uint8_t) (t[5] >>  7);
+        r[13 * i + 9] |=  (uint8_t) (t[6] <<  6);
+        r[13 * i + 10]  =  (uint8_t) (t[6] >>  2);
+        r[13 * i + 11]  =  (uint8_t) (t[6] >> 10);
+        r[13 * i + 11] |=  (uint8_t) (t[7] <<  3);
+        r[13 * i + 12]  =  (uint8_t) (t[7] >>  5);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_unpack
+*
+* Description: Unpack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyt0_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        r->coeffs[8 * i + 0]  = a[13 * i + 0];
+        r->coeffs[8 * i + 0] |= (uint32_t)a[13 * i + 1] << 8;
+        r->coeffs[8 * i + 0] &= 0x1FFF;
+
+        r->coeffs[8 * i + 1]  = a[13 * i + 1] >> 5;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 2] << 3;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 3] << 11;
+        r->coeffs[8 * i + 1] &= 0x1FFF;
+
+        r->coeffs[8 * i + 2]  = a[13 * i + 3] >> 2;
+        r->coeffs[8 * i + 2] |= (uint32_t)a[13 * i + 4] << 6;
+        r->coeffs[8 * i + 2] &= 0x1FFF;
+
+        r->coeffs[8 * i + 3]  = a[13 * i + 4] >> 7;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 5] << 1;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 6] << 9;
+        r->coeffs[8 * i + 3] &= 0x1FFF;
+
+        r->coeffs[8 * i + 4]  = a[13 * i + 6] >> 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 7] << 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 8] << 12;
+        r->coeffs[8 * i + 4] &= 0x1FFF;
+
+        r->coeffs[8 * i + 5]  = a[13 * i + 8] >> 1;
+        r->coeffs[8 * i + 5] |= (uint32_t)a[13 * i + 9] << 7;
+        r->coeffs[8 * i + 5] &= 0x1FFF;
+
+        r->coeffs[8 * i + 6]  = a[13 * i + 9] >> 6;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 10] << 2;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 11] << 10;
+        r->coeffs[8 * i + 6] &= 0x1FFF;
+
+        r->coeffs[8 * i + 7]  = a[13 * i + 11] >> 3;
+        r->coeffs[8 * i + 7] |= (uint32_t)a[13 * i + 12] << 5;
+        r->coeffs[8 * i + 7] &= 0x1FFF;
+
+        r->coeffs[8 * i + 0] = (1 << (D - 1)) - r->coeffs[8 * i + 0];
+        r->coeffs[8 * i + 1] = (1 << (D - 1)) - r->coeffs[8 * i + 1];
+        r->coeffs[8 * i + 2] = (1 << (D - 1)) - r->coeffs[8 * i + 2];
+        r->coeffs[8 * i + 3] = (1 << (D - 1)) - r->coeffs[8 * i + 3];
+        r->coeffs[8 * i + 4] = (1 << (D - 1)) - r->coeffs[8 * i + 4];
+        r->coeffs[8 * i + 5] = (1 << (D - 1)) - r->coeffs[8 * i + 5];
+        r->coeffs[8 * i + 6] = (1 << (D - 1)) - r->coeffs[8 * i + 6];
+        r->coeffs[8 * i + 7] = (1 << (D - 1)) - r->coeffs[8 * i + 7];
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_pack
+*
+* Description: Bit-pack polynomial with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYZ_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyz_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[4];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 4; ++i) {
+        t[0] = GAMMA1 - a->coeffs[4 * i + 0];
+        t[1] = GAMMA1 - a->coeffs[4 * i + 1];
+        t[2] = GAMMA1 - a->coeffs[4 * i + 2];
+        t[3] = GAMMA1 - a->coeffs[4 * i + 3];
+
+        r[9 * i + 0]  = t[0];
+        r[9 * i + 1]  = t[0] >> 8;
+        r[9 * i + 2]  = t[0] >> 16;
+        r[9 * i + 2] |= t[1] << 2;
+        r[9 * i + 3]  = t[1] >> 6;
+        r[9 * i + 4]  = t[1] >> 14;
+        r[9 * i + 4] |= t[2] << 4;
+        r[9 * i + 5]  = t[2] >> 4;
+        r[9 * i + 6]  = t[2] >> 12;
+        r[9 * i + 6] |= t[3] << 6;
+        r[9 * i + 7]  = t[3] >> 2;
+        r[9 * i + 8]  = t[3] >> 10;
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_unpack
+*
+* Description: Unpack polynomial z with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyz_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 4; ++i) {
+        r->coeffs[4 * i + 0]  = a[9 * i + 0];
+        r->coeffs[4 * i + 0] |= (uint32_t)a[9 * i + 1] << 8;
+        r->coeffs[4 * i + 0] |= (uint32_t)a[9 * i + 2] << 16;
+        r->coeffs[4 * i + 0] &= 0x3FFFF;
+
+        r->coeffs[4 * i + 1]  = a[9 * i + 2] >> 2;
+        r->coeffs[4 * i + 1] |= (uint32_t)a[9 * i + 3] << 6;
+        r->coeffs[4 * i + 1] |= (uint32_t)a[9 * i + 4] << 14;
+        r->coeffs[4 * i + 1] &= 0x3FFFF;
+
+        r->coeffs[4 * i + 2]  = a[9 * i + 4] >> 4;
+        r->coeffs[4 * i + 2] |= (uint32_t)a[9 * i + 5] << 4;
+        r->coeffs[4 * i + 2] |= (uint32_t)a[9 * i + 6] << 12;
+        r->coeffs[4 * i + 2] &= 0x3FFFF;
+
+        r->coeffs[4 * i + 3]  = a[9 * i + 6] >> 6;
+        r->coeffs[4 * i + 3] |= (uint32_t)a[9 * i + 7] << 2;
+        r->coeffs[4 * i + 3] |= (uint32_t)a[9 * i + 8] << 10;
+        r->coeffs[4 * i + 3] &= 0x3FFFF;
+
+        r->coeffs[4 * i + 0] = GAMMA1 - r->coeffs[4 * i + 0];
+        r->coeffs[4 * i + 1] = GAMMA1 - r->coeffs[4 * i + 1];
+        r->coeffs[4 * i + 2] = GAMMA1 - r->coeffs[4 * i + 2];
+        r->coeffs[4 * i + 3] = GAMMA1 - r->coeffs[4 * i + 3];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyw1_pack
+*
+* Description: Bit-pack polynomial w1 with coefficients in [0,15] or [0,43].
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYW1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyw1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 4; ++i) {
+        r[3 * i + 0]  = a->coeffs[4 * i + 0];
+        r[3 * i + 0] |= a->coeffs[4 * i + 1] << 6;
+        r[3 * i + 1]  = a->coeffs[4 * i + 1] >> 2;
+        r[3 * i + 1] |= a->coeffs[4 * i + 2] << 4;
+        r[3 * i + 2]  = a->coeffs[4 * i + 2] >> 4;
+        r[3 * i + 2] |= a->coeffs[4 * i + 3] << 2;
+    }
+
+
+    DBENCH_STOP(*tpack);
+}

src/sign/dilithium/dilithium2/aarch64/poly.h

@@ -0,0 +1,92 @@
+#ifndef POLY_H
+#define POLY_H
+#include "params.h"
+#include <stdint.h>
+
+typedef struct {
+    int32_t coeffs[N];
+} poly;
+
+#define poly_reduce DILITHIUM_NAMESPACE(poly_reduce)
+void poly_reduce(poly *a);
+#define poly_caddq DILITHIUM_NAMESPACE(poly_caddq)
+void poly_caddq(poly *a);
+#define poly_freeze DILITHIUM_NAMESPACE(poly_freeze)
+void poly_freeze(poly *a);
+
+#define poly_add DILITHIUM_NAMESPACE(poly_add)
+void poly_add(poly *c, const poly *a, const poly *b);
+#define poly_sub DILITHIUM_NAMESPACE(poly_sub)
+void poly_sub(poly *c, const poly *a, const poly *b);
+#define poly_shiftl DILITHIUM_NAMESPACE(poly_shiftl)
+void poly_shiftl(poly *a);
+
+#define poly_ntt DILITHIUM_NAMESPACE(poly_ntt)
+void poly_ntt(poly *a);
+#define poly_invntt_tomont DILITHIUM_NAMESPACE(poly_invntt_tomont)
+void poly_invntt_tomont(poly *a);
+#define poly_pointwise_montgomery DILITHIUM_NAMESPACE(poly_pointwise_montgomery)
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b);
+
+#define poly_power2round DILITHIUM_NAMESPACE(poly_power2round)
+void poly_power2round(poly *a1, poly *a0, const poly *a);
+#define poly_decompose DILITHIUM_NAMESPACE(poly_decompose)
+void poly_decompose(poly *a1, poly *a0, const poly *a);
+#define poly_make_hint DILITHIUM_NAMESPACE(poly_make_hint)
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1);
+#define poly_use_hint DILITHIUM_NAMESPACE(poly_use_hint)
+void poly_use_hint(poly *b, const poly *a, const poly *h);
+
+#define poly_chknorm DILITHIUM_NAMESPACE(poly_chknorm)
+int poly_chknorm(const poly *a, int32_t B);
+#define poly_uniform DILITHIUM_NAMESPACE(poly_uniform)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce);
+#define poly_uniformx2 DILITHIUM_NAMESPACE(poly_uniformx2)
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_eta DILITHIUM_NAMESPACE(poly_uniform_eta)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce);
+#define poly_uniform_etax2 DILITHIUM_NAMESPACE(poly_uniform_etax2)
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_gamma1 DILITHIUM_NAMESPACE(poly_uniform_gamma1)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce);
+#define poly_uniform_gamma1x2 DILITHIUM_NAMESPACE(poly_uniform_gamma1x2)
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1);
+#define poly_challenge DILITHIUM_NAMESPACE(poly_challenge)
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define polyeta_pack DILITHIUM_NAMESPACE(polyeta_pack)
+void polyeta_pack(uint8_t *r, const poly *a);
+#define polyeta_unpack DILITHIUM_NAMESPACE(polyeta_unpack)
+void polyeta_unpack(poly *r, const uint8_t *a);
+
+#define polyt1_pack DILITHIUM_NAMESPACE(polyt1_pack)
+void polyt1_pack(uint8_t *r, const poly *a);
+#define polyt1_unpack DILITHIUM_NAMESPACE(polyt1_unpack)
+void polyt1_unpack(poly *r, const uint8_t *a);
+
+#define polyt0_pack DILITHIUM_NAMESPACE(polyt0_pack)
+void polyt0_pack(uint8_t *r, const poly *a);
+#define polyt0_unpack DILITHIUM_NAMESPACE(polyt0_unpack)
+void polyt0_unpack(poly *r, const uint8_t *a);
+
+#define polyz_pack DILITHIUM_NAMESPACE(polyz_pack)
+void polyz_pack(uint8_t *r, const poly *a);
+#define polyz_unpack DILITHIUM_NAMESPACE(polyz_unpack)
+void polyz_unpack(poly *r, const uint8_t *a);
+
+#define polyw1_pack DILITHIUM_NAMESPACE(polyw1_pack)
+void polyw1_pack(uint8_t *r, const poly *a);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/polyvec.c

@@ -0,0 +1,452 @@
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#include "reduce.h"
+
+static const int32_t l_montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+/*************************************************
+* Name:        expand_mat
+*
+* Description: Implementation of ExpandA. Generates matrix A with uniformly
+*              random coefficients a_{i,j} by performing rejection
+*              sampling on the output stream of SHAKE128(rho|j|i)
+*              or AES256CTR(rho,j|i).
+*
+* Arguments:   - polyvecl mat[K]: output matrix
+*              - const uint8_t rho[]: byte array containing seed rho
+**************************************************/
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]) {
+    unsigned int i, j;
+
+    for (j = 0; j < L; ++j) {
+        for (i = 0; i < K; i += 2) {
+            poly_uniformx2(&mat[i + 0].vec[j], &mat[i + 1].vec[j], rho, (uint16_t) ((i << 8) + j), (uint16_t) (((i + 1) << 8) + j));
+        }
+    }
+}
+
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyvecl_pointwise_acc_montgomery(&t->vec[i], &mat[i], v);
+    }
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length L **************/
+/**************************************************************/
+
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+}
+
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L - 1; i += 2) {
+        poly_uniform_gamma1x2(&v->vec[i + 0], &v->vec[i + 1], seed, (uint16_t) (L * nonce + i + 0), (uint16_t) (L * nonce + i + 1));
+    }
+    if (L & 1) {
+        poly_uniform_gamma1(&v->vec[i], seed, (uint16_t) (L * nonce + L - 1));
+    }
+}
+
+void polyvecl_reduce(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length L
+*              to standard representatives.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_freeze(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_add
+*
+* Description: Add vectors of polynomials of length L.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyvecl *w: pointer to output vector
+*              - const polyvecl *u: pointer to first summand
+*              - const polyvecl *v: pointer to second summand
+**************************************************/
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length L. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_ntt(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+void polyvecl_invntt_tomont(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_pointwise_acc_montgomery
+*
+* Description: Pointwise multiply vectors of polynomials of length L, multiply
+*              resulting vector by 2^{-32} and add (accumulate) polynomials
+*              in it. Input/output vectors are in NTT domain representation.
+*
+* Arguments:   - poly *w: output polynomial
+*              - const polyvecl *u: pointer to first input vector
+*              - const polyvecl *v: pointer to second input vector
+**************************************************/
+extern void PQCLEAN_DILITHIUM2_AARCH64_asm_polyvecl_pointwise_acc_montgomery(int32_t *, const int32_t *, const int32_t *, const int32_t *);
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v) {
+    PQCLEAN_DILITHIUM2_AARCH64_asm_polyvecl_pointwise_acc_montgomery(w->coeffs, u->vec[0].coeffs, v->vec[0].coeffs, l_montgomery_const);
+}
+
+/*************************************************
+* Name:        polyvecl_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length L.
+*              Assumes input polyvecl to be reduced by polyvecl_reduce().
+*
+* Arguments:   - const polyvecl *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials is strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyvecl_chknorm(const polyvecl *v, int32_t bound)  {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length K **************/
+/**************************************************************/
+
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+
+}
+
+/*************************************************
+* Name:        polyveck_reduce
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to representatives in [-6283009,6283007].
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_reduce(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_caddq
+*
+* Description: For all coefficients of polynomials in vector of length K
+*              add Q if coefficient is negative.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_caddq(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_caddq(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to standard representatives.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_freeze(polyveck *v)  {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_add
+*
+* Description: Add vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first summand
+*              - const polyveck *v: pointer to second summand
+**************************************************/
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_sub
+*
+* Description: Subtract vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first input vector
+*              - const polyveck *v: pointer to second input vector to be
+*                                   subtracted from first input vector
+**************************************************/
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_sub(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_shiftl
+*
+* Description: Multiply vector of polynomials of Length K by 2^D without modular
+*              reduction. Assumes input coefficients to be less than 2^{31-D}.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_shiftl(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_shiftl(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length K. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_ntt(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_invntt_tomont
+*
+* Description: Inverse NTT and multiplication by 2^{32} of polynomials
+*              in vector of length K. Input coefficients need to be less
+*              than 2*Q.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_invntt_tomont(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+
+/*************************************************
+* Name:        polyveck_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length K.
+*              Assumes input polyveck to be reduced by polyveck_reduce().
+*
+* Arguments:   - const polyveck *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials are strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyveck_chknorm(const polyveck *v, int32_t bound) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        polyveck_power2round
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute a0, a1 such that a mod^+ Q = a1*2^D + a0
+*              with -2^{D-1} < a0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_power2round(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_decompose
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute high and low bits a0, a1 such a mod^+ Q = a1*ALPHA + a0
+*              with -ALPHA/2 < a0 <= ALPHA/2 except a1 = (Q-1)/ALPHA where we
+*              set a1 = 0 and -ALPHA/2 <= a0 = a mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_decompose(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_make_hint
+*
+* Description: Compute hint vector.
+*
+* Arguments:   - polyveck *h: pointer to output vector
+*              - const polyveck *v0: pointer to low part of input vector
+*              - const polyveck *v1: pointer to high part of input vector
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1) {
+    unsigned int i, s = 0;
+
+    for (i = 0; i < K; ++i) {
+        s += poly_make_hint(&h->vec[i], &v0->vec[i], &v1->vec[i]);
+    }
+
+    return s;
+}
+
+/*************************************************
+* Name:        polyveck_use_hint
+*
+* Description: Use hint vector to correct the high bits of input vector.
+*
+* Arguments:   - polyveck *w: pointer to output vector of polynomials with
+*                             corrected high bits
+*              - const polyveck *u: pointer to input vector
+*              - const polyveck *h: pointer to input hint vector
+**************************************************/
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_use_hint(&w->vec[i], &u->vec[i], &h->vec[i]);
+    }
+}
+
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyw1_pack(&r[i * POLYW1_PACKEDBYTES], &w1->vec[i]);
+    }
+}

src/sign/dilithium/dilithium2/aarch64/polyvec.h

@@ -0,0 +1,96 @@
+#ifndef POLYVEC_H
+#define POLYVEC_H
+#include "params.h"
+#include "poly.h"
+#include <stdint.h>
+
+/* Vectors of polynomials of length L */
+typedef struct {
+    poly vec[L];
+} polyvecl;
+
+#define polyvecl_uniform_eta DILITHIUM_NAMESPACE(polyvecl_uniform_eta)
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_uniform_gamma1 DILITHIUM_NAMESPACE(polyvecl_uniform_gamma1)
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_reduce DILITHIUM_NAMESPACE(polyvecl_reduce)
+void polyvecl_reduce(polyvecl *v);
+
+#define polyvecl_freeze DILITHIUM_NAMESPACE(polyvecl_freeze)
+void polyvecl_freeze(polyvecl *v);
+
+#define polyvecl_add DILITHIUM_NAMESPACE(polyvecl_add)
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v);
+
+#define polyvecl_ntt DILITHIUM_NAMESPACE(polyvecl_ntt)
+void polyvecl_ntt(polyvecl *v);
+#define polyvecl_invntt_tomont DILITHIUM_NAMESPACE(polyvecl_invntt_tomont)
+void polyvecl_invntt_tomont(polyvecl *v);
+#define polyvecl_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_poly_montgomery)
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v);
+#define polyvecl_pointwise_acc_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_acc_montgomery)
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v);
+
+
+#define polyvecl_chknorm DILITHIUM_NAMESPACE(polyvecl_chknorm)
+int polyvecl_chknorm(const polyvecl *v, int32_t B);
+
+
+
+/* Vectors of polynomials of length K */
+typedef struct {
+    poly vec[K];
+} polyveck;
+
+#define polyveck_uniform_eta DILITHIUM_NAMESPACE(polyveck_uniform_eta)
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyveck_reduce DILITHIUM_NAMESPACE(polyveck_reduce)
+void polyveck_reduce(polyveck *v);
+#define polyveck_caddq DILITHIUM_NAMESPACE(polyveck_caddq)
+void polyveck_caddq(polyveck *v);
+#define polyveck_freeze DILITHIUM_NAMESPACE(polyveck_freeze)
+void polyveck_freeze(polyveck *v);
+
+#define polyveck_add DILITHIUM_NAMESPACE(polyveck_add)
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_sub DILITHIUM_NAMESPACE(polyveck_sub)
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_shiftl DILITHIUM_NAMESPACE(polyveck_shiftl)
+void polyveck_shiftl(polyveck *v);
+
+#define polyveck_ntt DILITHIUM_NAMESPACE(polyveck_ntt)
+void polyveck_ntt(polyveck *v);
+#define polyveck_invntt_tomont DILITHIUM_NAMESPACE(polyveck_invntt_tomont)
+void polyveck_invntt_tomont(polyveck *v);
+#define polyveck_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyveck_pointwise_poly_montgomery)
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v);
+
+#define polyveck_chknorm DILITHIUM_NAMESPACE(polyveck_chknorm)
+int polyveck_chknorm(const polyveck *v, int32_t B);
+
+#define polyveck_power2round DILITHIUM_NAMESPACE(polyveck_power2round)
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_decompose DILITHIUM_NAMESPACE(polyveck_decompose)
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_make_hint DILITHIUM_NAMESPACE(polyveck_make_hint)
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1);
+#define polyveck_use_hint DILITHIUM_NAMESPACE(polyveck_use_hint)
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h);
+
+#define polyveck_pack_w1 DILITHIUM_NAMESPACE(polyveck_pack_w1)
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1);
+
+#define polyvec_matrix_expand DILITHIUM_NAMESPACE(polyvec_matrix_expand)
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]);
+
+#define polyvec_matrix_pointwise_montgomery DILITHIUM_NAMESPACE(polyvec_matrix_pointwise_montgomery)
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/reduce.c

@@ -0,0 +1,69 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        montgomery_reduce
+*
+* Description: For finite field element a with -2^{31}Q <= a <= Q*2^31,
+*              compute r \equiv a*2^{-32} (mod Q) such that -Q < r < Q.
+*
+* Arguments:   - int64_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t montgomery_reduce(int64_t a) {
+    int32_t t;
+
+    t = (int32_t)((uint64_t)a * (uint64_t)DILITHIUM_QINV);
+    t = (a - (int64_t)t * DILITHIUM_Q) >> 32;
+    return t;
+}
+
+/*************************************************
+* Name:        reduce32
+*
+* Description: For finite field element a with a <= 2^{31} - 2^{22} - 1,
+*              compute r \equiv a (mod Q) such that -6283009 <= r <= 6283007.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t reduce32(int32_t a) {
+    int32_t t;
+
+    t = (a + (1 << 22)) >> 23;
+    t = a - t * DILITHIUM_Q;
+    return t;
+}
+
+/*************************************************
+* Name:        caddq
+*
+* Description: Add Q if input coefficient is negative.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t caddq(int32_t a) {
+    a += (a >> 31) & DILITHIUM_Q;
+    return a;
+}
+
+/*************************************************
+* Name:        freeze
+*
+* Description: For finite field element a, compute standard
+*              representative r = a mod^+ Q.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t freeze(int32_t a) {
+    a = reduce32(a);
+    a = caddq(a);
+    return a;
+}

src/sign/dilithium/dilithium2/aarch64/reduce.h

@@ -0,0 +1,20 @@
+#ifndef REDUCE_H
+#define REDUCE_H
+#include "params.h"
+#include <stdint.h>
+
+#define DILITHIUM_QINV 58728449 // q^(-1) mod 2^32
+
+#define montgomery_reduce DILITHIUM_NAMESPACE(montgomery_reduce)
+int32_t montgomery_reduce(int64_t a);
+
+#define reduce32 DILITHIUM_NAMESPACE(reduce32)
+int32_t reduce32(int32_t a);
+
+#define caddq DILITHIUM_NAMESPACE(caddq)
+int32_t caddq(int32_t a);
+
+#define freeze DILITHIUM_NAMESPACE(freeze)
+int32_t freeze(int32_t a);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/rounding.c

@@ -0,0 +1,96 @@
+#include "params.h"
+#include "rounding.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        power2round
+*
+* Description: For finite field element a, compute a0, a1 such that
+*              a mod^+ Q = a1*2^D + a0 with -2^{D-1} < a0 <= 2^{D-1}.
+*              Assumes a to be standard representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t power2round(int32_t *a0, int32_t a)  {
+    int32_t a1;
+
+    a1 = (a + (1 << (D - 1)) - 1) >> D;
+    *a0 = a - (a1 << D);
+    return a1;
+}
+
+/*************************************************
+* Name:        decompose
+*
+* Description: For finite field element a, compute high and low bits a0, a1 such
+*              that a mod^+ Q = a1*ALPHA + a0 with -ALPHA/2 < a0 <= ALPHA/2 except
+*              if a1 = (Q-1)/ALPHA where we set a1 = 0 and
+*              -ALPHA/2 <= a0 = a mod^+ Q - Q < 0. Assumes a to be standard
+*              representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t decompose(int32_t *a0, int32_t a) {
+    int32_t a1;
+
+    a1  = (a + 127) >> 7;
+
+    a1  = (a1 * 11275 + (1 << 23)) >> 24;
+    a1 ^= ((43 - a1) >> 31) & a1;
+
+
+    *a0  = a - a1 * 2 * GAMMA2;
+    *a0 -= (((DILITHIUM_Q - 1) / 2 - *a0) >> 31) & DILITHIUM_Q;
+    return a1;
+}
+
+/*************************************************
+* Name:        make_hint
+*
+* Description: Compute hint bit indicating whether the low bits of the
+*              input element overflow into the high bits.
+*
+* Arguments:   - int32_t a0: low bits of input element
+*              - int32_t a1: high bits of input element
+*
+* Returns 1 if overflow.
+**************************************************/
+unsigned int make_hint(int32_t a0, int32_t a1) {
+    if (a0 > GAMMA2 || a0 < -GAMMA2 || (a0 == -GAMMA2 && a1 != 0)) {
+        return 1;
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        use_hint
+*
+* Description: Correct high bits according to hint.
+*
+* Arguments:   - int32_t a: input element
+*              - unsigned int hint: hint bit
+*
+* Returns corrected high bits.
+**************************************************/
+int32_t use_hint(int32_t a, unsigned int hint) {
+    int32_t a0, a1;
+
+    a1 = decompose(&a0, a);
+    if (hint == 0) {
+        return a1;
+    }
+
+
+    if (a0 > 0) {
+        return (a1 == 43) ?  0 : a1 + 1;
+    }
+    return (a1 ==  0) ? 43 : a1 - 1;
+
+}

src/sign/dilithium/dilithium2/aarch64/rounding.h

@@ -0,0 +1,18 @@
+#ifndef ROUNDING_H
+#define ROUNDING_H
+#include "params.h"
+#include <stdint.h>
+
+#define power2round DILITHIUM_NAMESPACE(power2round)
+int32_t power2round(int32_t *a0, int32_t a);
+
+#define decompose DILITHIUM_NAMESPACE(decompose)
+int32_t decompose(int32_t *a0, int32_t a);
+
+#define make_hint DILITHIUM_NAMESPACE(make_hint)
+unsigned int make_hint(int32_t a0, int32_t a1);
+
+#define use_hint DILITHIUM_NAMESPACE(use_hint)
+int32_t use_hint(int32_t a, unsigned int hint);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/sign.c

@@ -0,0 +1,343 @@
+#include "fips202.h"
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include "randombytes.h"
+#include "sign.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        crypto_sign_keypair
+*
+* Description: Generates public and private key.
+*
+* Arguments:   - uint8_t *pk: pointer to output public key (allocated
+*                             array of CRYPTO_PUBLICKEYBYTES bytes)
+*              - uint8_t *sk: pointer to output private key (allocated
+*                             array of CRYPTO_SECRETKEYBYTES bytes)
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk) {
+    uint8_t seedbuf[2 * SEEDBYTES + CRHBYTES];
+    uint8_t tr[SEEDBYTES];
+    const uint8_t *rho, *rhoprime, *key;
+    polyvecl mat[K];
+    polyvecl s1, s1hat;
+    polyveck s2, t1, t0;
+
+    /* Get randomness for rho, rhoprime and key */
+    randombytes(seedbuf, SEEDBYTES);
+    shake256(seedbuf, 2 * SEEDBYTES + CRHBYTES, seedbuf, SEEDBYTES);
+    rho = seedbuf;
+    rhoprime = rho + SEEDBYTES;
+    key = rhoprime + CRHBYTES;
+
+    /* Expand matrix */
+    polyvec_matrix_expand(mat, rho);
+
+    /* Sample short vectors s1 and s2 */
+    polyvecl_uniform_eta(&s1, rhoprime, 0);
+    polyveck_uniform_eta(&s2, rhoprime, L);
+
+    /* Matrix-vector multiplication */
+    s1hat = s1;
+    polyvecl_ntt(&s1hat);
+    polyvec_matrix_pointwise_montgomery(&t1, mat, &s1hat);
+    polyveck_reduce(&t1);
+    polyveck_invntt_tomont(&t1);
+
+    /* Add error vector s2 */
+    polyveck_add(&t1, &t1, &s2);
+
+    /* Extract t1 and write public key */
+    polyveck_caddq(&t1);
+    polyveck_power2round(&t1, &t0, &t1);
+    pack_pk(pk, rho, &t1);
+
+    /* Compute H(rho, t1) and write secret key */
+    shake256(tr, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    pack_sk(sk, rho, tr, key, &t0, &s1, &s2);
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_signature
+*
+* Description: Computes signature.
+*
+* Arguments:   - uint8_t *sig:   pointer to output signature (of length CRYPTO_BYTES)
+*              - size_t *siglen: pointer to output length of signature
+*              - uint8_t *m:     pointer to message to be signed
+*              - size_t mlen:    length of message
+*              - uint8_t *sk:    pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_signature(uint8_t *sig,
+                          size_t *siglen,
+                          const uint8_t *m,
+                          size_t mlen,
+                          const uint8_t *sk) {
+    unsigned int n;
+    uint8_t seedbuf[3 * SEEDBYTES + 2 * CRHBYTES];
+    uint8_t *rho, *tr, *key, *mu, *rhoprime;
+    uint16_t nonce = 0;
+    polyvecl mat[K], s1, y, z;
+    polyveck t0, s2, w1, w0, h;
+    poly cp;
+    shake256incctx state;
+
+    rho = seedbuf;
+    tr = rho + SEEDBYTES;
+    key = tr + SEEDBYTES;
+    mu = key + SEEDBYTES;
+    rhoprime = mu + CRHBYTES;
+    unpack_sk(rho, tr, key, &t0, &s1, &s2, sk);
+
+    /* Compute CRH(tr, msg) */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, tr, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    shake256(rhoprime, CRHBYTES, key, SEEDBYTES + CRHBYTES);
+
+    /* Expand matrix and transform vectors */
+    polyvec_matrix_expand(mat, rho);
+    polyvecl_ntt(&s1);
+    polyveck_ntt(&s2);
+    polyveck_ntt(&t0);
+
+rej:
+    /* Sample intermediate vector y */
+    polyvecl_uniform_gamma1(&y, rhoprime, nonce++);
+
+    /* Matrix-vector multiplication */
+    z = y;
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Decompose w and call the random oracle */
+    polyveck_caddq(&w1);
+    polyveck_decompose(&w1, &w0, &w1);
+    polyveck_pack_w1(sig, &w1);
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, sig, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(sig, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    poly_challenge(&cp, sig);
+    poly_ntt(&cp);
+
+    /* Compute z, reject if it reveals secret */
+    polyvecl_pointwise_poly_montgomery(&z, &cp, &s1);
+    polyvecl_invntt_tomont(&z);
+    polyvecl_add(&z, &z, &y);
+    polyvecl_reduce(&z);
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        goto rej;
+    }
+
+    /* Check that subtracting cs2 does not change high bits of w and low bits
+     * do not reveal secret information */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &s2);
+    polyveck_invntt_tomont(&h);
+    polyveck_sub(&w0, &w0, &h);
+    polyveck_reduce(&w0);
+    if (polyveck_chknorm(&w0, GAMMA2 - BETA)) {
+        goto rej;
+    }
+
+    /* Compute hints for w1 */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &t0);
+    polyveck_invntt_tomont(&h);
+    polyveck_reduce(&h);
+    if (polyveck_chknorm(&h, GAMMA2)) {
+        goto rej;
+    }
+
+    polyveck_add(&w0, &w0, &h);
+    n = polyveck_make_hint(&h, &w0, &w1);
+    if (n > OMEGA) {
+        goto rej;
+    }
+
+    /* Write signature */
+    pack_sig(sig, sig, &z, &h);
+    *siglen = CRYPTO_BYTES;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign
+*
+* Description: Compute signed message.
+*
+* Arguments:   - uint8_t *sm: pointer to output signed message (allocated
+*                             array with CRYPTO_BYTES + mlen bytes),
+*                             can be equal to m
+*              - size_t *smlen: pointer to output length of signed
+*                               message
+*              - const uint8_t *m: pointer to message to be signed
+*              - size_t mlen: length of message
+*              - const uint8_t *sk: pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign(uint8_t *sm,
+                size_t *smlen,
+                const uint8_t *m,
+                size_t mlen,
+                const uint8_t *sk) {
+    size_t i;
+
+    for (i = 0; i < mlen; ++i) {
+        sm[CRYPTO_BYTES + mlen - 1 - i] = m[mlen - 1 - i];
+    }
+    crypto_sign_signature(sm, smlen, sm + CRYPTO_BYTES, mlen, sk);
+    *smlen += mlen;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_verify
+*
+* Description: Verifies signature.
+*
+* Arguments:   - uint8_t *m: pointer to input signature
+*              - size_t siglen: length of signature
+*              - const uint8_t *m: pointer to message
+*              - size_t mlen: length of message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signature could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_verify(const uint8_t *sig,
+                       size_t siglen,
+                       const uint8_t *m,
+                       size_t mlen,
+                       const uint8_t *pk) {
+    unsigned int i;
+    uint8_t buf[K * POLYW1_PACKEDBYTES];
+    uint8_t rho[SEEDBYTES];
+    uint8_t mu[CRHBYTES];
+    uint8_t c[SEEDBYTES];
+    uint8_t c2[SEEDBYTES];
+    poly cp;
+    polyvecl mat[K], z;
+    polyveck t1, w1, h;
+    shake256incctx state;
+
+    if (siglen != CRYPTO_BYTES) {
+        return -1;
+    }
+
+    unpack_pk(rho, &t1, pk);
+    if (unpack_sig(c, &z, &h, sig)) {
+        return -1;
+    }
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        return -1;
+    }
+
+    /* Compute CRH(H(rho, t1), msg) */
+    shake256(mu, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    /* Matrix-vector multiplication; compute Az - c2^dt1 */
+    poly_challenge(&cp, c);
+    polyvec_matrix_expand(mat, rho);
+
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+
+    poly_ntt(&cp);
+    polyveck_shiftl(&t1);
+    polyveck_ntt(&t1);
+    polyveck_pointwise_poly_montgomery(&t1, &cp, &t1);
+
+    polyveck_sub(&w1, &w1, &t1);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Reconstruct w1 */
+    polyveck_caddq(&w1);
+    polyveck_use_hint(&w1, &w1, &h);
+    polyveck_pack_w1(buf, &w1);
+
+    /* Call random oracle and verify challenge */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, buf, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(c2, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    for (i = 0; i < SEEDBYTES; ++i) {
+        if (c[i] != c2[i]) {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_open
+*
+* Description: Verify signed message.
+*
+* Arguments:   - uint8_t *m: pointer to output message (allocated
+*                            array with smlen bytes), can be equal to sm
+*              - size_t *mlen: pointer to output length of message
+*              - const uint8_t *sm: pointer to signed message
+*              - size_t smlen: length of signed message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signed message could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_open(uint8_t *m,
+                     size_t *mlen,
+                     const uint8_t *sm,
+                     size_t smlen,
+                     const uint8_t *pk) {
+    size_t i;
+
+    if (smlen < CRYPTO_BYTES) {
+        goto badsig;
+    }
+
+    *mlen = smlen - CRYPTO_BYTES;
+    if (crypto_sign_verify(sm, CRYPTO_BYTES, sm + CRYPTO_BYTES, *mlen, pk)) {
+        goto badsig;
+    } else {
+        /* All good, copy msg, return 0 */
+        for (i = 0; i < *mlen; ++i) {
+            m[i] = sm[CRYPTO_BYTES + i];
+        }
+        return 0;
+    }
+
+badsig:
+    /* Signature verification failed */
+    *mlen = (size_t) -1;
+    for (i = 0; i < smlen; ++i) {
+        m[i] = 0;
+    }
+
+    return -1;
+}

src/sign/dilithium/dilithium2/aarch64/sign.h

@@ -0,0 +1,37 @@
+#ifndef SIGN_H
+#define SIGN_H
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stddef.h>
+#include <stdint.h>
+
+
+
+#define challenge DILITHIUM_NAMESPACE(challenge)
+void challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define crypto_sign_keypair DILITHIUM_NAMESPACE(crypto_sign_keypair)
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+#define crypto_sign_signature DILITHIUM_NAMESPACE(crypto_sign_signature)
+int crypto_sign_signature(uint8_t *sig, size_t *siglen,
+                          const uint8_t *m, size_t mlen,
+                          const uint8_t *sk);
+
+#define crypto_sign DILITHIUM_NAMESPACETOP
+int crypto_sign(uint8_t *sm, size_t *smlen,
+                const uint8_t *m, size_t mlen,
+                const uint8_t *sk);
+
+#define crypto_sign_verify DILITHIUM_NAMESPACE(crypto_sign_verify)
+int crypto_sign_verify(const uint8_t *sig, size_t siglen,
+                       const uint8_t *m, size_t mlen,
+                       const uint8_t *pk);
+
+#define crypto_sign_open DILITHIUM_NAMESPACE(crypto_sign_open)
+int crypto_sign_open(uint8_t *m, size_t *mlen,
+                     const uint8_t *sm, size_t smlen,
+                     const uint8_t *pk);
+
+#endif

src/sign/dilithium/dilithium2/aarch64/symmetric-shake.c

@@ -0,0 +1,66 @@
+#include "fips202.h"
+#include "params.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+void dilithium_shake128_stream_init(shake128incctx *state, const uint8_t seed[SEEDBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake128_inc_init(state);
+    shake128_inc_absorb(state, seed, SEEDBYTES);
+    shake128_inc_absorb(state, t, 2);
+    shake128_inc_finalize(state);
+}
+
+void dilithium_shake256_stream_init(shake256incctx *state, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake256_inc_init(state);
+    shake256_inc_absorb(state, seed, CRHBYTES);
+    shake256_inc_absorb(state, t, 2);
+    shake256_inc_finalize(state);
+}
+
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[SEEDBYTES + 2 + 14];
+    uint8_t extseed2[SEEDBYTES + 2 + 14];
+
+    for (i = 0; i < SEEDBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[SEEDBYTES] = (uint8_t) nonce1;
+    extseed1[SEEDBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[SEEDBYTES  ] = (uint8_t) nonce2;
+    extseed2[SEEDBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake128x2_absorb(state, extseed1, extseed2, SEEDBYTES + 2);
+}
+
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[CRHBYTES + 2 + 14];
+    uint8_t extseed2[CRHBYTES + 2 + 14];
+
+    for (i = 0; i < CRHBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[CRHBYTES] = (uint8_t) nonce1;
+    extseed1[CRHBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[CRHBYTES  ] = (uint8_t) nonce2;
+    extseed2[CRHBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake256x2_absorb(state, extseed1, extseed2, CRHBYTES + 2);
+}

src/sign/dilithium/dilithium2/aarch64/symmetric.h

@@ -0,0 +1,47 @@
+#ifndef SYMMETRIC_H
+#define SYMMETRIC_H
+#include "fips202.h"
+#include "fips202x2.h"
+#include "params.h"
+#include <stdint.h>
+
+
+typedef shake128incctx stream128_state;
+typedef shake256incctx stream256_state;
+
+#define dilithium_shake128_stream_init DILITHIUM_NAMESPACE(dilithium_shake128_stream_init)
+void dilithium_shake128_stream_init(shake128incctx *state,
+                                    const uint8_t seed[SEEDBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake256_stream_init DILITHIUM_NAMESPACE(dilithium_shake256_stream_init)
+void dilithium_shake256_stream_init(shake256incctx *state,
+                                    const uint8_t seed[CRHBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake128x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake128x2_stream_init)
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+#define dilithium_shake256x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake256x2_stream_init)
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+
+
+#define STREAM128_BLOCKBYTES SHAKE128_RATE
+#define STREAM256_BLOCKBYTES SHAKE256_RATE
+
+#define stream128_init(STATE, SEED, NONCE) \
+    dilithium_shake128_stream_init(STATE, SEED, NONCE)
+#define stream128_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake128_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE128_RATE), STATE)
+#define stream128_release(STATE) shake128_inc_ctx_release(STATE)
+#define stream256_init(STATE, SEED, NONCE) \
+    dilithium_shake256_stream_init(STATE, SEED, NONCE)
+#define stream256_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake256_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE256_RATE), STATE)
+#define stream256_release(STATE) shake256_inc_ctx_release(STATE)
+
+
+#endif

src/sign/dilithium/dilithium3/aarch64/LICENSE

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.

src/sign/dilithium/dilithium3/aarch64/Makefile

@@ -0,0 +1,23 @@
+# This Makefile can be used with GNU Make or BSD Make
+
+LIB=libdilithium3_aarch64.a
+HEADERS=api.h fips202x2.h macros_common.inc macros.inc NTT_params.h ntt.h packing.h params.h poly.h polyvec.h reduce.h rounding.h sign.h symmetric.h
+OBJECTS=fips202x2.o ntt.o packing.o poly.o polyvec.o reduce.o rounding.o sign.o symmetric-shake.o __asm_iNTT.o __asm_NTT.o __asm_poly.o
+
+
+CFLAGS=-O3 -Wall -Wextra -Wpedantic -Werror -Wmissing-prototypes -Wredundant-decls -std=c99 -I../../../common $(EXTRAFLAGS) -g
+
+all: $(LIB)
+
+%.o: %.c $(HEADERS)
+	$(CC) $(CFLAGS) -c -o $@ $<
+
+%.o: %.S $(HEADERS)
+	$(CC) $(CFLAGS) -c -o $@ $<
+
+$(LIB): $(OBJECTS) $(HEADERS)
+	$(AR) -r $@ $(OBJECTS)
+
+clean:
+	$(RM) $(OBJECTS)
+	$(RM) $(LIB)

src/sign/dilithium/dilithium3/aarch64/NTT_params.h

@@ -0,0 +1,56 @@
+#ifndef NTT_PARAMS_H
+#define NTT_PARAMS_H
+
+#define ARRAY_N 256
+
+#define NTT_N 256
+#define LOGNTT_N 8
+
+// root of unity: 1753
+
+
+// Q1
+#define Q1 8380417
+// omegaQ1 = 1753 mod Q1
+#define omegaQ1 1753
+// invomegaQ1 = omegaQ^{-1} mod Q1
+#define invomegaQ1 731434
+// R = 2^32 below
+// RmodQ1 = 2^32 mod^{+-} Q1
+#define RmodQ1 (-4186625)
+// Q1prime = Q1^{-1} mod^{+-} 2^32
+#define Q1prime 58728449
+// invNQ1 = NTT_N^{-1} mod Q1
+#define invNQ1 8347681
+
+// invNQ1R2modQ1 = -NTT_N^{-1} 2^32 2^32 mod^{+-} Q1 below
+#define invNQ1R2modQ1 (-41978)
+// invNQ1R2modQ1_prime = invNQ1R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime 8395782
+// invNQ1R2modQ1_prime_half = (invNQ1R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime_half 4197891
+// invNQ1R2modQ1_doubleprime = (invNQ1R2modQ1_prime Q1 - invNQ1R2modQ1) / 2^32
+#define invNQ1R2modQ1_doubleprime 16382
+
+// invNQ1_final_R2modQ1 = -invNQ1R2modQ1 invomegaQ1^{128} mod q
+#define invNQ1_final_R2modQ1 4404704
+// invNQ1_final_R2modQ1_prime = invNQ1_final_R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime (-151046688)
+// invNQ1_final_R2modQ1_prime_half = (invNQ1_final_R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime_half (-75523344)
+// invNQ1_final_R2modQ1_doubleprime = (invNQ1_final_R2modQ1_prime Q1 - invNQ1_final_R2modQ1) / 2^32
+#define invNQ1_final_R2modQ1_doubleprime (-294725)
+
+// RmodQ1_prime = -(RmodQ1 + Q1) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime 512
+// RmodQ1_prime_half = ( -(RmodQ1 + Q1) / 2) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime_half 256
+// RmodQ1_doubleprime = (RmodQ1_prime Q1 - RmodQ1_prime ) / 2^32
+#define RmodQ1_doubleprime 1
+
+#endif
+
+
+
+
+

src/sign/dilithium/dilithium3/aarch64/__asm_NTT.S

@@ -0,0 +1,274 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_top
+.global _PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_top:
+_PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    src1      .req x1
+    src2      .req x2
+    src3      .req x3
+    src4      .req x4
+    src5      .req x5
+    src6      .req x6
+    src7      .req x7
+    src8      .req x8
+    src9      .req x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v1.4S}, [ src1]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v7.4S}, [ src7]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v11.4S}, [src11]
+    ld1 {v13.4S}, [src13]
+    ld1 {v15.4S}, [src15]
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v8.4S}, [ src8]
+    ld1 {v10.4S}, [src10]
+    ld1 {v12.4S}, [src12]
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    mov counter, #3
+    _ntt_top_loop:
+
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_top_loop
+
+    st1 { v1.4S}, [ src1], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v7.4S}, [ src7], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v15.4S}, [src15], #16
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v8.4S}, [ src8], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v14.4S}, [src14], #16
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_bot
+.global _PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_bot:
+_PQCLEAN_DILITHIUM3_AARCH64_asm_ntt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _ntt_bot_loop:
+
+    ld1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld1 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+
+    dq_butterfly_top  v0,  v1,  v2,  v3, v12, v13, v4,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v20, 2, 3, v20, 2, 3,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_bot v16, v18, v17, v19, v28, v29, v4, v21, 0, 1, v21, 2, 3
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_vec_top  v0,  v1,  v2,  v3, v12, v13, v4,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v22, v23, v22, v23,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_bot v16, v18, v17, v19, v28, v29, v4, v24, v25, v26, v27
+
+    st4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st4 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_bot_loop
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+
+
+
+
+
+
+
+
+

src/sign/dilithium/dilithium3/aarch64/__asm_iNTT.S

@@ -0,0 +1,509 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_top
+.global _PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_top:
+_PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    Qhalf     .req w21
+    nQhalf    .req w22
+    invNR2ph  .req w24
+    invNR2dp  .req w25
+    invNWR2ph .req w26
+    invNWR2dp .req w27
+    src0      .req  x0
+    src1      .req  x1
+    src2      .req  x2
+    src3      .req  x3
+    src4      .req  x4
+    src5      .req  x5
+    src6      .req  x6
+    src7      .req  x7
+    src8      .req  x8
+    src9      .req  x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2, #0]
+    lsr Qhalf, Q, #1
+    neg nQhalf, Qhalf
+
+    ldr invNR2ph,  [x2, #16]
+    ldr invNR2dp,  [x2, #20]
+    ldr invNWR2ph, [x2, #24]
+    ldr invNWR2dp, [x2, #28]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v1.4S}, [ src1]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v7.4S}, [ src7]
+
+    ld1 { v8.4S}, [ src8]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v10.4S}, [src10]
+    ld1 {v11.4S}, [src11]
+    ld1 {v12.4S}, [src12]
+    ld1 {v13.4S}, [src13]
+    ld1 {v14.4S}, [src14]
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    mov counter, #3
+    _intt_top_loop:
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    sub counter, counter, #1
+    cbnz counter, _intt_top_loop
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v1.4S}, [ src1], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v7.4S}, [ src7], #16
+
+    st1 { v8.4S}, [ src8], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v14.4S}, [src14], #16
+    st1 {v15.4S}, [src15], #16
+
+    .unreq    Q
+    .unreq    Qhalf
+    .unreq    nQhalf
+    .unreq    invNR2ph
+    .unreq    invNR2dp
+    .unreq    invNWR2ph
+    .unreq    invNWR2dp
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_bot
+.global _PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_bot:
+_PQCLEAN_DILITHIUM3_AARCH64_asm_intt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    RphRdp    .req x21
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+    ldr RphRdp, [x2, #8]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512    
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _intt_bot_loop:
+
+    ld4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld4 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+    mov v20.D[0], RphRdp
+
+    dq_butterfly_vec_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v24, v25, v26, v27,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_top v16, v17, v28, v29, v18, v19,  v4, v22, v23, v22, v23
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v21, 0, 1, v21, 2, 3,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_top v16, v17, v28, v29, v18, v19,  v4, v20, 2, 3, v20, 2, 3
+
+    srshr v14.4S,  v0.4S, #23
+    srshr v15.4S,  v1.4S, #23
+    srshr v30.4S, v16.4S, #23
+    srshr v31.4S, v17.4S, #23
+
+    mls    v0.4S, v14.4S, v4.S[0]
+    mls    v1.4S, v15.4S, v4.S[0]
+    mls   v16.4S, v30.4S, v4.S[0]
+    mls   v17.4S, v31.4S, v4.S[0]
+
+    st1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st1 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _intt_bot_loop
+
+    .unreq    Q
+    .unreq    RphRdp
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+

src/sign/dilithium/dilithium3/aarch64/api.h

@@ -0,0 +1,32 @@
+#ifndef PQCLEAN_DILITHIUM3_AARCH64_API_H
+#define PQCLEAN_DILITHIUM3_AARCH64_API_H
+
+
+#include <stddef.h>
+#include <stdint.h>
+
+#define PQCLEAN_DILITHIUM3_AARCH64_CRYPTO_PUBLICKEYBYTES 1952
+#define PQCLEAN_DILITHIUM3_AARCH64_CRYPTO_SECRETKEYBYTES 4000
+#define PQCLEAN_DILITHIUM3_AARCH64_CRYPTO_BYTES 3293
+#define PQCLEAN_DILITHIUM3_AARCH64_CRYPTO_ALGNAME "Dilithium3"
+
+int PQCLEAN_DILITHIUM3_AARCH64_crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+int PQCLEAN_DILITHIUM3_AARCH64_crypto_sign_signature(
+    uint8_t *sig, size_t *siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM3_AARCH64_crypto_sign_verify(
+    const uint8_t *sig, size_t siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *pk);
+
+int PQCLEAN_DILITHIUM3_AARCH64_crypto_sign(
+    uint8_t *sm, size_t *smlen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM3_AARCH64_crypto_sign_open(
+    uint8_t *m, size_t *mlen,
+    const uint8_t *sm, size_t smlen, const uint8_t *pk);
+
+
+#endif

src/sign/dilithium/dilithium3/aarch64/fips202x2.c

@@ -0,0 +1,646 @@
+#include "fips202x2.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+
+#define NROUNDS 24
+
+// Define NEON operation
+// c = load(ptr)
+#define vload(ptr) vld1q_u64(ptr);
+// ptr <= c;
+#define vstore(ptr, c) vst1q_u64(ptr, c);
+// c = a ^ b
+#define vxor(c, a, b) c = veorq_u64(a, b);
+// Rotate by n bit ((a << offset) ^ (a >> (64-offset)))
+#define vROL(out, a, offset)    \
+    (out) = vshlq_n_u64(a, offset); \
+    (out) = vsriq_n_u64(out, a, 64 - (offset));
+// Xor chain: out = a ^ b ^ c ^ d ^ e
+#define vXOR4(out, a, b, c, d, e) \
+    (out) = veorq_u64(a, b);          \
+    (out) = veorq_u64(out, c);        \
+    (out) = veorq_u64(out, d);        \
+    (out) = veorq_u64(out, e);
+// Not And c = ~a & b
+// #define vbic(c, a, b) c = vbicq_u64(b, a);
+// Xor Not And: out = a ^ ( (~b) & c)
+#define vXNA(out, a, b, c) \
+    (out) = vbicq_u64(c, b);   \
+    (out) = veorq_u64(out, a);
+// Rotate by 1 bit, then XOR: a ^ ROL(b): SHA1 instruction, not support
+#define vrxor(c, a, b) c = vrax1q_u64(a, b);
+// End Define
+
+/* Keccak round constants */
+static const uint64_t neon_KeccakF_RoundConstants[NROUNDS] = {
+    (uint64_t)0x0000000000000001ULL,
+    (uint64_t)0x0000000000008082ULL,
+    (uint64_t)0x800000000000808aULL,
+    (uint64_t)0x8000000080008000ULL,
+    (uint64_t)0x000000000000808bULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008009ULL,
+    (uint64_t)0x000000000000008aULL,
+    (uint64_t)0x0000000000000088ULL,
+    (uint64_t)0x0000000080008009ULL,
+    (uint64_t)0x000000008000000aULL,
+    (uint64_t)0x000000008000808bULL,
+    (uint64_t)0x800000000000008bULL,
+    (uint64_t)0x8000000000008089ULL,
+    (uint64_t)0x8000000000008003ULL,
+    (uint64_t)0x8000000000008002ULL,
+    (uint64_t)0x8000000000000080ULL,
+    (uint64_t)0x000000000000800aULL,
+    (uint64_t)0x800000008000000aULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008080ULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008008ULL
+};
+
+/*************************************************
+* Name:        KeccakF1600_StatePermutex2
+*
+* Description: The Keccak F1600 Permutation
+*
+* Arguments:   - uint64_t *state: pointer to input/output Keccak state
+**************************************************/
+static inline
+void KeccakF1600_StatePermutex2(v128 state[25]) {
+    v128 Aba, Abe, Abi, Abo, Abu;
+    v128 Aga, Age, Agi, Ago, Agu;
+    v128 Aka, Ake, Aki, Ako, Aku;
+    v128 Ama, Ame, Ami, Amo, Amu;
+    v128 Asa, Ase, Asi, Aso, Asu;
+    v128 BCa, BCe, BCi, BCo, BCu; // tmp
+    v128 Da, De, Di, Do, Du;      // D
+    v128 Eba, Ebe, Ebi, Ebo, Ebu;
+    v128 Ega, Ege, Egi, Ego, Egu;
+    v128 Eka, Eke, Eki, Eko, Eku;
+    v128 Ema, Eme, Emi, Emo, Emu;
+    v128 Esa, Ese, Esi, Eso, Esu;
+
+    //copyFromState(A, state)
+    Aba = state[0];
+    Abe = state[1];
+    Abi = state[2];
+    Abo = state[3];
+    Abu = state[4];
+    Aga = state[5];
+    Age = state[6];
+    Agi = state[7];
+    Ago = state[8];
+    Agu = state[9];
+    Aka = state[10];
+    Ake = state[11];
+    Aki = state[12];
+    Ako = state[13];
+    Aku = state[14];
+    Ama = state[15];
+    Ame = state[16];
+    Ami = state[17];
+    Amo = state[18];
+    Amu = state[19];
+    Asa = state[20];
+    Ase = state[21];
+    Asi = state[22];
+    Aso = state[23];
+    Asu = state[24];
+
+    for (int round = 0; round < NROUNDS; round += 2) {
+        //    prepareTheta
+        vXOR4(BCa, Aba, Aga, Aka, Ama, Asa);
+        vXOR4(BCe, Abe, Age, Ake, Ame, Ase);
+        vXOR4(BCi, Abi, Agi, Aki, Ami, Asi);
+        vXOR4(BCo, Abo, Ago, Ako, Amo, Aso);
+        vXOR4(BCu, Abu, Agu, Aku, Amu, Asu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round  , A, E)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Aba, Aba, Da);
+        vxor(Age, Age, De);
+        vROL(BCe, Age, 44);
+        vxor(Aki, Aki, Di);
+        vROL(BCi, Aki, 43);
+        vxor(Amo, Amo, Do);
+        vROL(BCo, Amo, 21);
+        vxor(Asu, Asu, Du);
+        vROL(BCu, Asu, 14);
+        vXNA(Eba, Aba, BCe, BCi);
+        vxor(Eba, Eba, vdupq_n_u64(neon_KeccakF_RoundConstants[round]));
+        vXNA(Ebe, BCe, BCi, BCo);
+        vXNA(Ebi, BCi, BCo, BCu);
+        vXNA(Ebo, BCo, BCu, Aba);
+        vXNA(Ebu, BCu, Aba, BCe);
+
+        vxor(Abo, Abo, Do);
+        vROL(BCa, Abo, 28);
+        vxor(Agu, Agu, Du);
+        vROL(BCe, Agu, 20);
+        vxor(Aka, Aka, Da);
+        vROL(BCi, Aka, 3);
+        vxor(Ame, Ame, De);
+        vROL(BCo, Ame, 45);
+        vxor(Asi, Asi, Di);
+        vROL(BCu, Asi, 61);
+        vXNA(Ega, BCa, BCe, BCi);
+        vXNA(Ege, BCe, BCi, BCo);
+        vXNA(Egi, BCi, BCo, BCu);
+        vXNA(Ego, BCo, BCu, BCa);
+        vXNA(Egu, BCu, BCa, BCe);
+
+        vxor(Abe, Abe, De);
+        vROL(BCa, Abe, 1);
+        vxor(Agi, Agi, Di);
+        vROL(BCe, Agi, 6);
+        vxor(Ako, Ako, Do);
+        vROL(BCi, Ako, 25);
+        vxor(Amu, Amu, Du);
+        vROL(BCo, Amu, 8);
+        vxor(Asa, Asa, Da);
+        vROL(BCu, Asa, 18);
+        vXNA(Eka, BCa, BCe, BCi);
+        vXNA(Eke, BCe, BCi, BCo);
+        vXNA(Eki, BCi, BCo, BCu);
+        vXNA(Eko, BCo, BCu, BCa);
+        vXNA(Eku, BCu, BCa, BCe);
+
+        vxor(Abu, Abu, Du);
+        vROL(BCa, Abu, 27);
+        vxor(Aga, Aga, Da);
+        vROL(BCe, Aga, 36);
+        vxor(Ake, Ake, De);
+        vROL(BCi, Ake, 10);
+        vxor(Ami, Ami, Di);
+        vROL(BCo, Ami, 15);
+        vxor(Aso, Aso, Do);
+        vROL(BCu, Aso, 56);
+        vXNA(Ema, BCa, BCe, BCi);
+        vXNA(Eme, BCe, BCi, BCo);
+        vXNA(Emi, BCi, BCo, BCu);
+        vXNA(Emo, BCo, BCu, BCa);
+        vXNA(Emu, BCu, BCa, BCe);
+
+        vxor(Abi, Abi, Di);
+        vROL(BCa, Abi, 62);
+        vxor(Ago, Ago, Do);
+        vROL(BCe, Ago, 55);
+        vxor(Aku, Aku, Du);
+        vROL(BCi, Aku, 39);
+        vxor(Ama, Ama, Da);
+        vROL(BCo, Ama, 41);
+        vxor(Ase, Ase, De);
+        vROL(BCu, Ase, 2);
+        vXNA(Esa, BCa, BCe, BCi);
+        vXNA(Ese, BCe, BCi, BCo);
+        vXNA(Esi, BCi, BCo, BCu);
+        vXNA(Eso, BCo, BCu, BCa);
+        vXNA(Esu, BCu, BCa, BCe);
+
+        // Next Round
+
+        //    prepareTheta
+        vXOR4(BCa, Eba, Ega, Eka, Ema, Esa);
+        vXOR4(BCe, Ebe, Ege, Eke, Eme, Ese);
+        vXOR4(BCi, Ebi, Egi, Eki, Emi, Esi);
+        vXOR4(BCo, Ebo, Ego, Eko, Emo, Eso);
+        vXOR4(BCu, Ebu, Egu, Eku, Emu, Esu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round+1, E, A)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Eba, Eba, Da);
+        vxor(Ege, Ege, De);
+        vROL(BCe, Ege, 44);
+        vxor(Eki, Eki, Di);
+        vROL(BCi, Eki, 43);
+        vxor(Emo, Emo, Do);
+        vROL(BCo, Emo, 21);
+        vxor(Esu, Esu, Du);
+        vROL(BCu, Esu, 14);
+        vXNA(Aba, Eba, BCe, BCi);
+        vxor(Aba, Aba, vdupq_n_u64(neon_KeccakF_RoundConstants[round + 1]));
+        vXNA(Abe, BCe, BCi, BCo);
+        vXNA(Abi, BCi, BCo, BCu);
+        vXNA(Abo, BCo, BCu, Eba);
+        vXNA(Abu, BCu, Eba, BCe);
+
+        vxor(Ebo, Ebo, Do);
+        vROL(BCa, Ebo, 28);
+        vxor(Egu, Egu, Du);
+        vROL(BCe, Egu, 20);
+        vxor(Eka, Eka, Da);
+        vROL(BCi, Eka, 3);
+        vxor(Eme, Eme, De);
+        vROL(BCo, Eme, 45);
+        vxor(Esi, Esi, Di);
+        vROL(BCu, Esi, 61);
+        vXNA(Aga, BCa, BCe, BCi);
+        vXNA(Age, BCe, BCi, BCo);
+        vXNA(Agi, BCi, BCo, BCu);
+        vXNA(Ago, BCo, BCu, BCa);
+        vXNA(Agu, BCu, BCa, BCe);
+
+        vxor(Ebe, Ebe, De);
+        vROL(BCa, Ebe, 1);
+        vxor(Egi, Egi, Di);
+        vROL(BCe, Egi, 6);
+        vxor(Eko, Eko, Do);
+        vROL(BCi, Eko, 25);
+        vxor(Emu, Emu, Du);
+        vROL(BCo, Emu, 8);
+        vxor(Esa, Esa, Da);
+        vROL(BCu, Esa, 18);
+        vXNA(Aka, BCa, BCe, BCi);
+        vXNA(Ake, BCe, BCi, BCo);
+        vXNA(Aki, BCi, BCo, BCu);
+        vXNA(Ako, BCo, BCu, BCa);
+        vXNA(Aku, BCu, BCa, BCe);
+
+        vxor(Ebu, Ebu, Du);
+        vROL(BCa, Ebu, 27);
+        vxor(Ega, Ega, Da);
+        vROL(BCe, Ega, 36);
+        vxor(Eke, Eke, De);
+        vROL(BCi, Eke, 10);
+        vxor(Emi, Emi, Di);
+        vROL(BCo, Emi, 15);
+        vxor(Eso, Eso, Do);
+        vROL(BCu, Eso, 56);
+        vXNA(Ama, BCa, BCe, BCi);
+        vXNA(Ame, BCe, BCi, BCo);
+        vXNA(Ami, BCi, BCo, BCu);
+        vXNA(Amo, BCo, BCu, BCa);
+        vXNA(Amu, BCu, BCa, BCe);
+
+        vxor(Ebi, Ebi, Di);
+        vROL(BCa, Ebi, 62);
+        vxor(Ego, Ego, Do);
+        vROL(BCe, Ego, 55);
+        vxor(Eku, Eku, Du);
+        vROL(BCi, Eku, 39);
+        vxor(Ema, Ema, Da);
+        vROL(BCo, Ema, 41);
+        vxor(Ese, Ese, De);
+        vROL(BCu, Ese, 2);
+        vXNA(Asa, BCa, BCe, BCi);
+        vXNA(Ase, BCe, BCi, BCo);
+        vXNA(Asi, BCi, BCo, BCu);
+        vXNA(Aso, BCo, BCu, BCa);
+        vXNA(Asu, BCu, BCa, BCe);
+    }
+
+    state[0] = Aba;
+    state[1] = Abe;
+    state[2] = Abi;
+    state[3] = Abo;
+    state[4] = Abu;
+    state[5] = Aga;
+    state[6] = Age;
+    state[7] = Agi;
+    state[8] = Ago;
+    state[9] = Agu;
+    state[10] = Aka;
+    state[11] = Ake;
+    state[12] = Aki;
+    state[13] = Ako;
+    state[14] = Aku;
+    state[15] = Ama;
+    state[16] = Ame;
+    state[17] = Ami;
+    state[18] = Amo;
+    state[19] = Amu;
+    state[20] = Asa;
+    state[21] = Ase;
+    state[22] = Asi;
+    state[23] = Aso;
+    state[24] = Asu;
+}
+
+/*************************************************
+* Name:        keccakx2_absorb
+*
+* Description: Absorb step of Keccak;
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - uint64_t *s: pointer to (uninitialized) output Keccak state
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - const uint8_t *m: pointer to input to be absorbed into s
+*              - size_t mlen: length of input in bytes
+*              - uint8_t p: domain-separation byte for different
+*                           Keccak-derived functions
+**************************************************/
+static
+void keccakx2_absorb(v128 s[25],
+                     unsigned int r,
+                     const uint8_t *in0,
+                     const uint8_t *in1,
+                     size_t inlen,
+                     uint8_t p) {
+    size_t i, pos = 0;
+
+    // Declare SIMD registers
+    v128 tmp, mask;
+    uint64x1_t a, b;
+    uint64x2_t a1, b1, atmp1, btmp1;
+    uint64x2x2_t a2, b2, atmp2, btmp2;
+    // End
+
+    for (i = 0; i < 25; ++i) {
+        s[i] = vdupq_n_u64(0);
+    }
+
+    // Load in0[i] to register, then in1[i] to register, exchange them
+    while (inlen >= r) {
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2 = vld1q_u64_x2((uint64_t *)&in0[pos]);
+            b2 = vld1q_u64_x2((uint64_t *)&in1[pos]);
+            // BD = zip1(AB and CD)
+            atmp2.val[0] = vzip1q_u64(a2.val[0], b2.val[0]);
+            atmp2.val[1] = vzip1q_u64(a2.val[1], b2.val[1]);
+            // AC = zip2(AB and CD)
+            btmp2.val[0] = vzip2q_u64(a2.val[0], b2.val[0]);
+            btmp2.val[1] = vzip2q_u64(a2.val[1], b2.val[1]);
+
+            vxor(s[i + 0], s[i + 0], atmp2.val[0]);
+            vxor(s[i + 1], s[i + 1], btmp2.val[0]);
+            vxor(s[i + 2], s[i + 2], atmp2.val[1]);
+            vxor(s[i + 3], s[i + 3], btmp2.val[1]);
+
+            pos += 8 * 2 * 2;
+        }
+        // Last iteration
+        i = r / 8 - 1;
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+        pos += 8;
+
+        KeccakF1600_StatePermutex2(s);
+        inlen -= r;
+    }
+
+    i = 0;
+    while (inlen >= 16) {
+        a1 = vld1q_u64((uint64_t *)&in0[pos]);
+        b1 = vld1q_u64((uint64_t *)&in1[pos]);
+        // BD = zip1(AB and CD)
+        atmp1 = vzip1q_u64(a1, b1);
+        // AC = zip2(AB and CD)
+        btmp1 = vzip2q_u64(a1, b1);
+
+        vxor(s[i + 0], s[i + 0], atmp1);
+        vxor(s[i + 1], s[i + 1], btmp1);
+
+        i += 2;
+        pos += 8 * 2;
+        inlen -= 8 * 2;
+    }
+
+    if (inlen >= 8) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+
+        i++;
+        pos += 8;
+        inlen -= 8;
+    }
+
+    if (inlen) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        mask = vdupq_n_u64((1ULL << (8 * inlen)) - 1);
+        tmp = vandq_u64(tmp, mask);
+        vxor(s[i], s[i], tmp);
+    }
+
+    tmp = vdupq_n_u64((uint64_t)p << (8 * inlen));
+    vxor(s[i], s[i], tmp);
+
+    mask = vdupq_n_u64(1ULL << 63);
+    vxor(s[r / 8 - 1], s[r / 8 - 1], mask);
+}
+
+/*************************************************
+* Name:        keccak_squeezeblocks
+*
+* Description: Squeeze step of Keccak. Squeezes full blocks of r bytes each.
+*              Modifies the state. Can be called multiple times to keep
+*              squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out: pointer to output blocks
+*              - size_t nblocks: number of blocks to be squeezed (written to h)
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - uint64_t *s: pointer to input/output Keccak state
+**************************************************/
+static
+void keccakx2_squeezeblocks(uint8_t *out0,
+                            uint8_t *out1,
+                            size_t nblocks,
+                            unsigned int r,
+                            v128 s[25]) {
+    unsigned int i;
+
+    uint64x1_t a, b;
+    uint64x2x2_t a2, b2;
+
+    while (nblocks > 0) {
+        KeccakF1600_StatePermutex2(s);
+
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2.val[0] = vuzp1q_u64(s[i], s[i + 1]);
+            b2.val[0] = vuzp2q_u64(s[i], s[i + 1]);
+            a2.val[1] = vuzp1q_u64(s[i + 2], s[i + 3]);
+            b2.val[1] = vuzp2q_u64(s[i + 2], s[i + 3]);
+            vst1q_u64_x2((uint64_t *)out0, a2);
+            vst1q_u64_x2((uint64_t *)out1, b2);
+
+            out0 += 32;
+            out1 += 32;
+        }
+
+        i = r / 8 - 1;
+        // Last iteration
+        a = vget_low_u64(s[i]);
+        b = vget_high_u64(s[i]);
+        vst1_u64((uint64_t *)out0, a);
+        vst1_u64((uint64_t *)out1, b);
+
+        out0 += 8;
+        out1 += 8;
+
+        --nblocks;
+    }
+}
+
+/*************************************************
+* Name:        shake128x2_absorb
+*
+* Description: Absorb step of the SHAKE128 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *state: pointer to (uninitialized) output
+*                                     Keccak state
+*              - const uint8_t *in:   pointer to input to be absorbed into s
+*              - size_t inlen:        length of input in bytes
+**************************************************/
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE128_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake128_squeezeblocks
+*
+* Description: Squeeze step of SHAKE128 XOF. Squeezes full blocks of
+*              SHAKE128_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE128_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake256_absorb
+*
+* Description: Absorb step of the SHAKE256 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *s:   pointer to (uninitialized) output Keccak state
+*              - const uint8_t *in: pointer to input to be absorbed into s
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE256_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake256_squeezeblocks
+*
+* Description: Squeeze step of SHAKE256 XOF. Squeezes full blocks of
+*              SHAKE256_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE256_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake128
+*
+* Description: SHAKE128 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE128_RATE;
+    uint8_t t[2][SHAKE128_RATE];
+    keccakx2_state state;
+
+    shake128x2_absorb(&state, in0, in1, inlen);
+    shake128x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE128_RATE;
+    out1 += nblocks * SHAKE128_RATE;
+    outlen -= nblocks * SHAKE128_RATE;
+
+    if (outlen) {
+        shake128x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}
+
+/*************************************************
+* Name:        shake256
+*
+* Description: SHAKE256 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE256_RATE;
+    uint8_t t[2][SHAKE256_RATE];
+    keccakx2_state state;
+
+    shake256x2_absorb(&state, in0, in1, inlen);
+    shake256x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE256_RATE;
+    out1 += nblocks * SHAKE256_RATE;
+    outlen -= nblocks * SHAKE256_RATE;
+
+    if (outlen) {
+        shake256x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}

src/sign/dilithium/dilithium3/aarch64/fips202x2.h

@@ -0,0 +1,60 @@
+#ifndef FIPS202X2_H
+#define FIPS202X2_H
+
+#include "params.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+typedef uint64x2_t v128;
+
+#define SHAKE128_RATE 168
+#define SHAKE256_RATE 136
+#define SHA3_256_RATE 136
+#define SHA3_512_RATE 72
+
+
+typedef struct {
+    v128 s[25];
+} keccakx2_state;
+
+
+#define shake128x2_absorb DILITHIUM_NAMESPACE(shake128x2_absorb)
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake128x2_squeezeblocks DILITHIUM_NAMESPACE(shake128x2_squeezeblocks)
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake256x2_absorb DILITHIUM_NAMESPACE(shake256x2_absorb)
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake256x2_squeezeblocks DILITHIUM_NAMESPACE(shake256x2_squeezeblocks)
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake128x2 DILITHIUM_NAMESPACE(shake128x2)
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+
+#define shake256x2 DILITHIUM_NAMESPACE(shake256x2)
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+#endif

src/sign/dilithium/dilithium3/aarch64/macros.inc

@@ -0,0 +1,84 @@
+#include "macros_common.inc"
+
+.macro wrap_trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3, qS, dD
+
+    trn1 \t0\qS, \a0\qS, \a1\qS
+    trn2 \t1\qS, \a0\qS, \a1\qS
+    trn1 \t2\qS, \a2\qS, \a3\qS
+    trn2 \t3\qS, \a2\qS, \a3\qS
+
+    trn1 \a0\dD, \t0\dD, \t2\dD
+    trn2 \a2\dD, \t0\dD, \t2\dD
+    trn1 \a1\dD, \t1\dD, \t3\dD
+    trn2 \a3\dD, \t1\dD, \t3\dD
+
+.endm
+
+.macro trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3
+    wrap_trn_4x4 \a0, \a1, \a2, \a3, \t0, \t1, \t2, \t3, .4S, .2D
+.endm
+
+
+.macro dq_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+
+.macro dq_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_montgomery_mul \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_top \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_bot \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed_rev \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+
+.macro qq_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q
+    wrap_qX_montgomery \c0, \c1, \c2, \c3, \l0, \l1, \l2, \l3, \h0, \h1, \h2, \h3, \t0, \t1, \t2, \t3, \Qprime, \Q, .2S, .4S, .2D
+.endm
+
+.macro qq_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3
+    wrap_qX_sub_add \s0, \s1, \s2, \s3, \t0, \t1, \t2, \t3, \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, .4S
+.endm

src/sign/dilithium/dilithium3/aarch64/macros_common.inc

@@ -0,0 +1,423 @@
+// for ABI
+
+.macro push_all
+
+    sub sp, sp, #(16*9)
+    stp x19, x20, [sp, #16*0]
+    stp x21, x22, [sp, #16*1]
+    stp x23, x24, [sp, #16*2]
+    stp x25, x26, [sp, #16*3]
+    stp x27, x28, [sp, #16*4]
+    stp  d8,  d9, [sp, #16*5]
+    stp d10, d11, [sp, #16*6]
+    stp d12, d13, [sp, #16*7]
+    stp d14, d15, [sp, #16*8]
+
+.endm
+
+.macro pop_all
+
+    ldp x19, x20, [sp, #16*0]
+    ldp x21, x22, [sp, #16*1]
+    ldp x23, x24, [sp, #16*2]
+    ldp x25, x26, [sp, #16*3]
+    ldp x27, x28, [sp, #16*4]
+    ldp  d8,  d9, [sp, #16*5]
+    ldp d10, d11, [sp, #16*6]
+    ldp d12, d13, [sp, #16*7]
+    ldp d14, d15, [sp, #16*8]
+    add sp, sp, #(16*9)
+
+.endm
+
+// vector-scalar butterflies
+
+.macro wrap_dX_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+    sub     \b2\wX, \a2\wX, \t2\wX
+    sub     \b3\wX, \a3\wX, \t3\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+    add     \a2\wX, \a2\wX, \t2\wX
+    add     \a3\wX, \a3\wX, \t3\wX
+
+.endm
+
+.macro wrap_qX_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t4\wX, \b4\wX, \z4\nX[\h4]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t5\wX, \b5\wX, \z5\nX[\h5]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t6\wX, \b6\wX, \z6\nX[\h6]
+    sub      \b3\wX, \a3\wX, \t3\wX
+    mul      \t7\wX, \b7\wX, \z7\nX[\h7]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b4\wX, \b4\wX, \z4\nX[\l4]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b5\wX, \b5\wX, \z5\nX[\l5]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b6\wX, \b6\wX, \z6\nX[\l6]
+    add      \a3\wX, \a3\wX, \t3\wX
+    sqrdmulh \b7\wX, \b7\wX, \z7\nX[\l7]
+
+    mls      \t4\wX, \b4\wX, \mod\nX[0]
+    mls      \t5\wX, \b5\wX, \mod\nX[0]
+    mls      \t6\wX, \b6\wX, \mod\nX[0]
+    mls      \t7\wX, \b7\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b4\wX, \a4\wX, \t4\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b5\wX, \a5\wX, \t5\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b6\wX, \a6\wX, \t6\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+    sub      \b7\wX, \a7\wX, \t7\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a4\wX, \a4\wX, \t4\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a5\wX, \a5\wX, \t5\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a6\wX, \a6\wX, \t6\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+    add      \a7\wX, \a7\wX, \t7\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+// vector-vector butterflies
+
+.macro wrap_dX_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \h2\wX
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \h3\wX
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \l2\wX
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \l3\wX
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+// vector-scalar Barrett reduction
+
+.macro wrap_qX_barrett a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oX_barrett a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\nX[0]
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\nX[0]
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\nX[0]
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// vector-vector Barrett reduction
+
+.macro wrap_qo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\wX
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\wX
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// Montgomery multiplication
+
+.macro wrap_qX_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \b0\wX, \t0\wX, \z0\nX[\h0]
+    mul      \b1\wX, \t1\wX, \z1\nX[\h1]
+    mul      \b2\wX, \t2\wX, \z2\nX[\h2]
+    mul      \b3\wX, \t3\wX, \z3\nX[\h3]
+
+    sqrdmulh \t0\wX, \t0\wX, \z0\nX[\l0]
+    sqrdmulh \t1\wX, \t1\wX, \z1\nX[\l1]
+    sqrdmulh \t2\wX, \t2\wX, \z2\nX[\l2]
+    sqrdmulh \t3\wX, \t3\wX, \z3\nX[\l3]
+
+    mls      \b0\wX, \t0\wX, \mod\nX[0]
+    mls      \b1\wX, \t1\wX, \mod\nX[0]
+    mls      \b2\wX, \t2\wX, \mod\nX[0]
+    mls      \b3\wX, \t3\wX, \mod\nX[0]
+
+.endm
+
+// Montgomery reduction with long
+
+.macro wrap_qX_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q, lX, wX, dwX
+
+    uzp1 \t0\wX, \l0\wX, \h0\wX
+    uzp1 \t1\wX, \l1\wX, \h1\wX
+    uzp1 \t2\wX, \l2\wX, \h2\wX
+    uzp1 \t3\wX, \l3\wX, \h3\wX
+
+    mul \t0\wX, \t0\wX, \Qprime\wX
+    mul \t1\wX, \t1\wX, \Qprime\wX
+    mul \t2\wX, \t2\wX, \Qprime\wX
+    mul \t3\wX, \t3\wX, \Qprime\wX
+
+    smlal  \l0\dwX, \t0\lX, \Q\lX
+    smlal2 \h0\dwX, \t0\wX, \Q\wX
+    smlal  \l1\dwX, \t1\lX, \Q\lX
+    smlal2 \h1\dwX, \t1\wX, \Q\wX
+    smlal  \l2\dwX, \t2\lX, \Q\lX
+    smlal2 \h2\dwX, \t2\wX, \Q\wX
+    smlal  \l3\dwX, \t3\lX, \Q\lX
+    smlal2 \h3\dwX, \t3\wX, \Q\wX
+
+    uzp2 \c0\wX, \l0\wX, \h0\wX
+    uzp2 \c1\wX, \l1\wX, \h1\wX
+    uzp2 \c2\wX, \l2\wX, \h2\wX
+    uzp2 \c3\wX, \l3\wX, \h3\wX
+
+.endm
+
+// add_sub, sub_add
+
+.macro wrap_qX_add_sub s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s3\wX, \a3\wX, \b3\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+
+.endm
+
+.macro wrap_qX_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+    add \s3\wX, \a3\wX, \b3\wX
+
+.endm

src/sign/dilithium/dilithium3/aarch64/ntt.c

@@ -0,0 +1,35 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+#include <stdio.h>
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+
+/*************************************************
+* Name:        ntt
+*
+* Description: Forward NTT, in-place. No modular reduction is performed after
+*              additions or subtractions. Output vector is in bitreversed order.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void ntt(int32_t a[N]) {
+    NTT(a);
+}
+
+/*************************************************
+* Name:        invntt_tomont
+*
+* Description: Inverse NTT and multiplication by Montgomery factor 2^32.
+*              In-place. No modular reductions after additions or
+*              subtractions; input coefficients need to be smaller than
+*              Q in absolute value. Output coefficient are smaller than Q in
+*              absolute value.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void invntt_tomont(int32_t a[N]) {
+    iNTT(a);
+}

src/sign/dilithium/dilithium3/aarch64/packing.c

@@ -0,0 +1,261 @@
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+
+
+/*************************************************
+* Name:        pack_pk
+*
+* Description: Bit-pack public key pk = (rho, t1).
+*
+* Arguments:   - uint8_t pk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const polyveck *t1: pointer to vector t1
+**************************************************/
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const polyveck *t1) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        pk[i] = rho[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_pack(pk + i * POLYT1_PACKEDBYTES, &t1->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_pk
+*
+* Description: Unpack public key pk = (rho, t1).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const polyveck *t1: pointer to output vector t1
+*              - uint8_t pk[]: byte array containing bit-packed pk
+**************************************************/
+void unpack_pk(uint8_t rho[SEEDBYTES],
+               polyveck *t1,
+               const uint8_t pk[CRYPTO_PUBLICKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = pk[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_unpack(&t1->vec[i], pk + i * POLYT1_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sk
+*
+* Description: Bit-pack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - uint8_t sk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const uint8_t tr[]: byte array containing tr
+*              - const uint8_t key[]: byte array containing key
+*              - const polyveck *t0: pointer to vector t0
+*              - const polyvecl *s1: pointer to vector s1
+*              - const polyveck *s2: pointer to vector s2
+**************************************************/
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = rho[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = key[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = tr[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s1->vec[i]);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s2->vec[i]);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_pack(sk + i * POLYT0_PACKEDBYTES, &t0->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_sk
+*
+* Description: Unpack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const uint8_t tr[]: output byte array for tr
+*              - const uint8_t key[]: output byte array for key
+*              - const polyveck *t0: pointer to output vector t0
+*              - const polyvecl *s1: pointer to output vector s1
+*              - const polyveck *s2: pointer to output vector s2
+*              - uint8_t sk[]: byte array containing bit-packed sk
+**************************************************/
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        key[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        tr[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_unpack(&s1->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_unpack(&s2->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_unpack(&t0->vec[i], sk + i * POLYT0_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sig
+*
+* Description: Bit-pack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t sig[]: output byte array
+*              - const uint8_t *c: pointer to challenge hash length SEEDBYTES
+*              - const polyvecl *z: pointer to vector z
+*              - const polyveck *h: pointer to hint vector h
+**************************************************/
+void pack_sig(uint8_t sig[CRYPTO_BYTES],
+              const uint8_t c[SEEDBYTES],
+              const polyvecl *z,
+              const polyveck *h) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sig[i] = c[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_pack(sig + i * POLYZ_PACKEDBYTES, &z->vec[i]);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Encode h */
+    for (i = 0; i < OMEGA + K; ++i) {
+        sig[i] = 0;
+    }
+
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            if (h->vec[i].coeffs[j] != 0) {
+                sig[k++] = (uint8_t) j;
+            }
+        }
+
+        sig[OMEGA + i] = (uint8_t) k;
+    }
+}
+
+/*************************************************
+* Name:        unpack_sig
+*
+* Description: Unpack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t *c: pointer to output challenge hash
+*              - polyvecl *z: pointer to output vector z
+*              - polyveck *h: pointer to output hint vector h
+*              - const uint8_t sig[]: byte array containing
+*                bit-packed signature
+*
+* Returns 1 in case of malformed signature; otherwise 0.
+**************************************************/
+int unpack_sig(uint8_t c[SEEDBYTES],
+               polyvecl *z,
+               polyveck *h,
+               const uint8_t sig[CRYPTO_BYTES]) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        c[i] = sig[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_unpack(&z->vec[i], sig + i * POLYZ_PACKEDBYTES);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Decode h */
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            h->vec[i].coeffs[j] = 0;
+        }
+
+        if (sig[OMEGA + i] < k || sig[OMEGA + i] > OMEGA) {
+            return 1;
+        }
+
+        for (j = k; j < sig[OMEGA + i]; ++j) {
+            /* Coefficients are ordered for strong unforgeability */
+            if (j > k && sig[j] <= sig[j - 1]) {
+                return 1;
+            }
+            h->vec[i].coeffs[sig[j]] = 1;
+        }
+
+        k = sig[OMEGA + i];
+    }
+
+    /* Extra indices are zero for strong unforgeability */
+    for (j = k; j < OMEGA; ++j) {
+        if (sig[j]) {
+            return 1;
+        }
+    }
+
+    return 0;
+}

src/sign/dilithium/dilithium3/aarch64/packing.h

@@ -0,0 +1,37 @@
+#ifndef PACKING_H
+#define PACKING_H
+#include "params.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#define pack_pk DILITHIUM_NAMESPACE(pack_pk)
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES], const uint8_t rho[SEEDBYTES], const polyveck *t1);
+
+#define pack_sk DILITHIUM_NAMESPACE(pack_sk)
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2);
+
+#define pack_sig DILITHIUM_NAMESPACE(pack_sig)
+void pack_sig(uint8_t sig[CRYPTO_BYTES], const uint8_t c[SEEDBYTES], const polyvecl *z, const polyveck *h);
+
+#define unpack_pk DILITHIUM_NAMESPACE(unpack_pk)
+void unpack_pk(uint8_t rho[SEEDBYTES], polyveck *t1, const uint8_t pk[CRYPTO_PUBLICKEYBYTES]);
+
+#define unpack_sk DILITHIUM_NAMESPACE(unpack_sk)
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]);
+
+#define unpack_sig DILITHIUM_NAMESPACE(unpack_sig)
+int unpack_sig(uint8_t c[SEEDBYTES], polyvecl *z, polyveck *h, const uint8_t sig[CRYPTO_BYTES]);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/params.h

@@ -0,0 +1,51 @@
+#ifndef PARAMS_H
+#define PARAMS_H
+
+//#define DILITHIUM_MODE 2
+#define DILITHIUM_MODE 3
+//#define DILITHIUM_MODE 5
+
+#define CRYPTO_NAMESPACETOP PQCLEAN_DILITHIUM3_AARCH64_crypto_sign
+#define CRYPTO_NAMESPACE(s) PQCLEAN_DILITHIUM3_AARCH64_##s
+#define DILITHIUM_NAMESPACETOP CRYPTO_NAMESPACETOP
+#define DILITHIUM_NAMESPACE(s) CRYPTO_NAMESPACE(s)
+
+
+#define SEEDBYTES 32
+#define CRHBYTES 64
+#define N 256
+#define DILITHIUM_Q 8380417
+#define D 13
+#define ROOT_OF_UNITY 1753
+
+
+#define K 6
+#define L 5
+#define ETA 4
+#define TAU 49
+#define BETA 196
+#define GAMMA1 (1 << 19)
+#define GAMMA2 ((DILITHIUM_Q-1)/32)
+#define OMEGA 55
+#define CRYPTO_ALGNAME "Dilithium3"
+
+
+#define POLYT1_PACKEDBYTES  320
+#define POLYT0_PACKEDBYTES  416
+#define POLYVECH_PACKEDBYTES (OMEGA + K)
+
+
+#define POLYZ_PACKEDBYTES   640
+
+#define POLYW1_PACKEDBYTES  128
+
+#define POLYETA_PACKEDBYTES 128
+
+#define CRYPTO_PUBLICKEYBYTES (SEEDBYTES + K*POLYT1_PACKEDBYTES)
+#define CRYPTO_SECRETKEYBYTES (3*SEEDBYTES \
+                               + L*POLYETA_PACKEDBYTES \
+                               + K*POLYETA_PACKEDBYTES \
+                               + K*POLYT0_PACKEDBYTES)
+#define CRYPTO_BYTES (SEEDBYTES + L*POLYZ_PACKEDBYTES + POLYVECH_PACKEDBYTES)
+
+#endif

src/sign/dilithium/dilithium3/aarch64/poly.c

@@ -0,0 +1,891 @@
+#include "params.h"
+#include "poly.h"
+#include "reduce.h"
+#include "rounding.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+#include "fips202x2.h"
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+static const int32_t montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+#define DBENCH_START()
+#define DBENCH_STOP(t)
+
+/*************************************************
+* Name:        poly_reduce
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              representative in [-6283009,6283007].
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_poly_reduce(int32_t *, const int32_t *);
+void poly_reduce(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_poly_reduce(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_caddq
+*
+* Description: For all coefficients of in/out polynomial add Q if
+*              coefficient is negative.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_poly_caddq(int32_t *, const int32_t *);
+void poly_caddq(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_poly_caddq(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_freeze
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              standard representatives.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_poly_freeze(int32_t *, const int32_t *);
+void poly_freeze(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_poly_freeze(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_add
+*
+* Description: Add polynomials. No modular reduction is performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first summand
+*              - const poly *b: pointer to second summand
+**************************************************/
+void poly_add(poly *c, const poly *a, const poly *b)  {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] + b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_sub
+*
+* Description: Subtract polynomials. No modular reduction is
+*              performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial to be
+*                               subtraced from first input polynomial
+**************************************************/
+void poly_sub(poly *c, const poly *a, const poly *b) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] - b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_shiftl
+*
+* Description: Multiply polynomial by 2^D without modular reduction. Assumes
+*              input coefficients to be less than 2^{31-D} in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_shiftl(poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a->coeffs[i] <<= D;
+    }
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_ntt
+*
+* Description: Inplace forward NTT. Coefficients can grow by
+*              8*Q in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_ntt(poly *a) {
+    DBENCH_START();
+
+    ntt(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_invntt_tomont
+*
+* Description: Inplace inverse NTT and multiplication by 2^{32}.
+*              Input coefficients need to be less than Q in absolute
+*              value and output coefficients are again bounded by Q.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_invntt_tomont(poly *a) {
+    DBENCH_START();
+
+    invntt_tomont(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_pointwise_montgomery
+*
+* Description: Pointwise multiplication of polynomials in NTT domain
+*              representation and multiplication of resulting polynomial
+*              by 2^{-32}.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_poly_pointwise_montgomery(int32_t *des, const int32_t *src1, const int32_t *src2, const int32_t *table);
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_poly_pointwise_montgomery(c->coeffs, a->coeffs, b->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_power2round
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute c0, c1 such that c mod Q = c1*2^D + c0
+*              with -2^{D-1} < c0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_poly_power2round(int32_t *, int32_t *, const int32_t *);
+void poly_power2round(poly *a1, poly *a0, const poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_poly_power2round(a1->coeffs, a0->coeffs, a->coeffs);
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_decompose
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute high and low bits c0, c1 such c mod Q = c1*ALPHA + c0
+*              with -ALPHA/2 < c0 <= ALPHA/2 except c1 = (Q-1)/ALPHA where we
+*              set c1 = 0 and -ALPHA/2 <= c0 = c mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void poly_decompose(poly *a1, poly *a0, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a1->coeffs[i] = decompose(&a0->coeffs[i], a->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_make_hint
+*
+* Description: Compute hint polynomial. The coefficients of which indicate
+*              whether the low bits of the corresponding coefficient of
+*              the input polynomial overflow into the high bits.
+*
+* Arguments:   - poly *h: pointer to output hint polynomial
+*              - const poly *a0: pointer to low part of input polynomial
+*              - const poly *a1: pointer to high part of input polynomial
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1) {
+    unsigned int i, s = 0;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        h->coeffs[i] = make_hint(a0->coeffs[i], a1->coeffs[i]);
+        s += h->coeffs[i];
+    }
+
+    DBENCH_STOP(*tround);
+    return s;
+}
+
+/*************************************************
+* Name:        poly_use_hint
+*
+* Description: Use hint polynomial to correct the high bits of a polynomial.
+*
+* Arguments:   - poly *b: pointer to output polynomial with corrected high bits
+*              - const poly *a: pointer to input polynomial
+*              - const poly *h: pointer to input hint polynomial
+**************************************************/
+void poly_use_hint(poly *b, const poly *a, const poly *h) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        b->coeffs[i] = use_hint(a->coeffs[i], h->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_chknorm
+*
+* Description: Check infinity norm of polynomial against given bound.
+*              Assumes input coefficients were reduced by reduce32().
+*
+* Arguments:   - const poly *a: pointer to polynomial
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm is strictly smaller than B <= (Q-1)/8 and 1 otherwise.
+**************************************************/
+int poly_chknorm(const poly *a, int32_t B) {
+    unsigned int i;
+    int32_t t;
+    DBENCH_START();
+
+    if (B > (DILITHIUM_Q - 1) / 8) {
+        return 1;
+    }
+
+    /* It is ok to leak which coefficient violates the bound since
+       the probability for each coefficient is independent of secret
+       data but we must not leak the sign of the centralized representative. */
+    for (i = 0; i < N; ++i) {
+        /* Absolute value */
+        t = a->coeffs[i] >> 31;
+        t = a->coeffs[i] - (t & 2 * a->coeffs[i]);
+
+        if (t >= B) {
+            DBENCH_STOP(*tsample);
+            return 1;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return 0;
+}
+
+/*************************************************
+* Name:        rej_uniform
+*
+* Description: Sample uniformly random coefficients in [0, Q-1] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_uniform(int32_t *a,
+                                unsigned int len,
+                                const uint8_t *buf,
+                                unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos + 3 <= buflen) {
+        t  = buf[pos++];
+        t |= (uint32_t)buf[pos++] << 8;
+        t |= (uint32_t)buf[pos++] << 16;
+        t &= 0x7FFFFF;
+
+        if (t < DILITHIUM_Q) {
+            a[ctr++] = t;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [0,Q-1] by performing rejection sampling on the
+*              output stream of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length SEEDBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+
+#define POLY_UNIFORM_NBLOCKS ((768 + STREAM128_BLOCKBYTES - 1)/STREAM128_BLOCKBYTES)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce) {
+    unsigned int i, ctr, off;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    stream128_state state;
+
+    stream128_init(&state, seed, nonce);
+    stream128_squeezeblocks(buf, POLY_UNIFORM_NBLOCKS, &state);
+
+    ctr = rej_uniform(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        off = buflen % 3;
+        for (i = 0; i < off; ++i) {
+            buf[i] = buf[buflen - off + i];
+        }
+
+        stream128_squeezeblocks(buf + off, 1, &state);
+        buflen = STREAM128_BLOCKBYTES + off;
+        ctr += rej_uniform(a->coeffs + ctr, N - ctr, buf, buflen);
+    }
+    stream128_release(&state);
+}
+
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf0[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    uint8_t buf1[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+
+    keccakx2_state statex2;
+    dilithium_shake128x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake128x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_NBLOCKS, &statex2);
+
+    ctr0 = rej_uniform(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_uniform(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake128x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_uniform(a0->coeffs + ctr0, N - ctr0, buf0, buflen);
+        ctr1 += rej_uniform(a1->coeffs + ctr1, N - ctr1, buf1, buflen);
+    }
+
+
+}
+
+/*************************************************
+* Name:        rej_eta
+*
+* Description: Sample uniformly random coefficients in [-ETA, ETA] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_eta(int32_t *a,
+                            unsigned int len,
+                            const uint8_t *buf,
+                            unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t0, t1;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos < buflen) {
+        t0 = buf[pos] & 0x0F;
+        t1 = buf[pos++] >> 4;
+
+
+        if (t0 < 9) {
+            a[ctr++] = 4 - t0;
+        }
+        if (t1 < 9 && ctr < len) {
+            a[ctr++] = 4 - t1;
+        }
+
+
+
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform_eta
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-ETA,ETA] by performing rejection sampling on the
+*              output stream from SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+#define POLY_UNIFORM_ETA_NBLOCKS ((227 + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce) {
+    unsigned int ctr;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_ETA_NBLOCKS, &state);
+
+    ctr = rej_eta(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        stream256_squeezeblocks(buf, 1, &state);
+        ctr += rej_eta(a->coeffs + ctr, N - ctr, buf, STREAM256_BLOCKBYTES);
+    }
+    stream256_release(&state);
+}
+
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+
+    uint8_t buf0[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_ETA_NBLOCKS, &statex2);
+
+    ctr0 = rej_eta(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_eta(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake256x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_eta(a0->coeffs + ctr0, N - ctr0, buf0, STREAM256_BLOCKBYTES);
+        ctr1 += rej_eta(a1->coeffs + ctr1, N - ctr1, buf1, STREAM256_BLOCKBYTES);
+    }
+}
+
+/*************************************************
+* Name:        poly_uniform_gamma1m1
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-(GAMMA1 - 1), GAMMA1] by unpacking output stream
+*              of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 16-bit nonce
+**************************************************/
+#define POLY_UNIFORM_GAMMA1_NBLOCKS ((POLYZ_PACKEDBYTES + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce) {
+    uint8_t buf[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_GAMMA1_NBLOCKS, &state);
+    stream256_release(&state);
+    polyz_unpack(a, buf);
+}
+
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1) {
+
+    uint8_t buf0[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_GAMMA1_NBLOCKS, &statex2);
+
+    polyz_unpack(a0, buf0);
+    polyz_unpack(a1, buf1);
+
+}
+
+/*************************************************
+* Name:        challenge
+*
+* Description: Implementation of H. Samples polynomial with TAU nonzero
+*              coefficients in {-1,1} using the output stream of
+*              SHAKE256(seed).
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const uint8_t mu[]: byte array containing seed of length SEEDBYTES
+**************************************************/
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]) {
+    unsigned int i, b, pos;
+    uint64_t signs;
+    uint8_t buf[SHAKE256_RATE];
+    shake256incctx state;
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, seed, SEEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(buf, sizeof buf, &state);
+
+    signs = 0;
+    for (i = 0; i < 8; ++i) {
+        signs |= (uint64_t)buf[i] << 8 * i;
+    }
+    pos = 8;
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = 0;
+    }
+    for (i = N - TAU; i < N; ++i) {
+        do {
+            if (pos >= SHAKE256_RATE) {
+                shake256_inc_squeeze(buf, sizeof buf, &state);
+                pos = 0;
+            }
+
+            b = buf[pos++];
+        } while (b > i);
+
+        c->coeffs[i] = c->coeffs[b];
+        c->coeffs[b] = 1 - 2 * (signs & 1);
+        signs >>= 1;
+    }
+    shake256_inc_ctx_release(&state);
+}
+
+/*************************************************
+* Name:        polyeta_pack
+*
+* Description: Bit-pack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYETA_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyeta_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint8_t t[8];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        t[0] = ETA - a->coeffs[2 * i + 0];
+        t[1] = ETA - a->coeffs[2 * i + 1];
+        r[i] = t[0] | (t[1] << 4);
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyeta_unpack
+*
+* Description: Unpack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyeta_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        r->coeffs[2 * i + 0] = a[i] & 0x0F;
+        r->coeffs[2 * i + 1] = a[i] >> 4;
+        r->coeffs[2 * i + 0] = ETA - r->coeffs[2 * i + 0];
+        r->coeffs[2 * i + 1] = ETA - r->coeffs[2 * i + 1];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_pack
+*
+* Description: Bit-pack polynomial t1 with coefficients fitting in 10 bits.
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 4; ++i) {
+        r[5 * i + 0] = (uint8_t) (a->coeffs[4 * i + 0] >> 0);
+        r[5 * i + 1] = (uint8_t) ((a->coeffs[4 * i + 0] >> 8) | (a->coeffs[4 * i + 1] << 2));
+        r[5 * i + 2] = (uint8_t) ((a->coeffs[4 * i + 1] >> 6) | (a->coeffs[4 * i + 2] << 4));
+        r[5 * i + 3] = (uint8_t) ((a->coeffs[4 * i + 2] >> 4) | (a->coeffs[4 * i + 3] << 6));
+        r[5 * i + 4] = (uint8_t) (a->coeffs[4 * i + 3] >> 2);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_unpack
+*
+* Description: Unpack polynomial t1 with 10-bit coefficients.
+*              Output coefficients are standard representatives.
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_10_to_32(int32_t *, const uint8_t *);
+void polyt1_unpack(poly *r, const uint8_t *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM3_AARCH64_asm_10_to_32(r->coeffs, a);
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_pack
+*
+* Description: Bit-pack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT0_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt0_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[8];
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        t[0] = (1 << (D - 1)) - a->coeffs[8 * i + 0];
+        t[1] = (1 << (D - 1)) - a->coeffs[8 * i + 1];
+        t[2] = (1 << (D - 1)) - a->coeffs[8 * i + 2];
+        t[3] = (1 << (D - 1)) - a->coeffs[8 * i + 3];
+        t[4] = (1 << (D - 1)) - a->coeffs[8 * i + 4];
+        t[5] = (1 << (D - 1)) - a->coeffs[8 * i + 5];
+        t[6] = (1 << (D - 1)) - a->coeffs[8 * i + 6];
+        t[7] = (1 << (D - 1)) - a->coeffs[8 * i + 7];
+
+        r[13 * i + 0]  =  (uint8_t) t[0];
+        r[13 * i + 1]  =  (uint8_t) (t[0] >>  8);
+        r[13 * i + 1] |=  (uint8_t) (t[1] <<  5);
+        r[13 * i + 2]  =  (uint8_t) (t[1] >>  3);
+        r[13 * i + 3]  =  (uint8_t) (t[1] >> 11);
+        r[13 * i + 3] |=  (uint8_t) (t[2] <<  2);
+        r[13 * i + 4]  =  (uint8_t) (t[2] >>  6);
+        r[13 * i + 4] |=  (uint8_t) (t[3] <<  7);
+        r[13 * i + 5]  =  (uint8_t) (t[3] >>  1);
+        r[13 * i + 6]  =  (uint8_t) (t[3] >>  9);
+        r[13 * i + 6] |=  (uint8_t) (t[4] <<  4);
+        r[13 * i + 7]  =  (uint8_t) (t[4] >>  4);
+        r[13 * i + 8]  =  (uint8_t) (t[4] >> 12);
+        r[13 * i + 8] |=  (uint8_t) (t[5] <<  1);
+        r[13 * i + 9]  =  (uint8_t) (t[5] >>  7);
+        r[13 * i + 9] |=  (uint8_t) (t[6] <<  6);
+        r[13 * i + 10]  =  (uint8_t) (t[6] >>  2);
+        r[13 * i + 11]  =  (uint8_t) (t[6] >> 10);
+        r[13 * i + 11] |=  (uint8_t) (t[7] <<  3);
+        r[13 * i + 12]  =  (uint8_t) (t[7] >>  5);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_unpack
+*
+* Description: Unpack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyt0_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        r->coeffs[8 * i + 0]  = a[13 * i + 0];
+        r->coeffs[8 * i + 0] |= (uint32_t)a[13 * i + 1] << 8;
+        r->coeffs[8 * i + 0] &= 0x1FFF;
+
+        r->coeffs[8 * i + 1]  = a[13 * i + 1] >> 5;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 2] << 3;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 3] << 11;
+        r->coeffs[8 * i + 1] &= 0x1FFF;
+
+        r->coeffs[8 * i + 2]  = a[13 * i + 3] >> 2;
+        r->coeffs[8 * i + 2] |= (uint32_t)a[13 * i + 4] << 6;
+        r->coeffs[8 * i + 2] &= 0x1FFF;
+
+        r->coeffs[8 * i + 3]  = a[13 * i + 4] >> 7;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 5] << 1;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 6] << 9;
+        r->coeffs[8 * i + 3] &= 0x1FFF;
+
+        r->coeffs[8 * i + 4]  = a[13 * i + 6] >> 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 7] << 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 8] << 12;
+        r->coeffs[8 * i + 4] &= 0x1FFF;
+
+        r->coeffs[8 * i + 5]  = a[13 * i + 8] >> 1;
+        r->coeffs[8 * i + 5] |= (uint32_t)a[13 * i + 9] << 7;
+        r->coeffs[8 * i + 5] &= 0x1FFF;
+
+        r->coeffs[8 * i + 6]  = a[13 * i + 9] >> 6;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 10] << 2;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 11] << 10;
+        r->coeffs[8 * i + 6] &= 0x1FFF;
+
+        r->coeffs[8 * i + 7]  = a[13 * i + 11] >> 3;
+        r->coeffs[8 * i + 7] |= (uint32_t)a[13 * i + 12] << 5;
+        r->coeffs[8 * i + 7] &= 0x1FFF;
+
+        r->coeffs[8 * i + 0] = (1 << (D - 1)) - r->coeffs[8 * i + 0];
+        r->coeffs[8 * i + 1] = (1 << (D - 1)) - r->coeffs[8 * i + 1];
+        r->coeffs[8 * i + 2] = (1 << (D - 1)) - r->coeffs[8 * i + 2];
+        r->coeffs[8 * i + 3] = (1 << (D - 1)) - r->coeffs[8 * i + 3];
+        r->coeffs[8 * i + 4] = (1 << (D - 1)) - r->coeffs[8 * i + 4];
+        r->coeffs[8 * i + 5] = (1 << (D - 1)) - r->coeffs[8 * i + 5];
+        r->coeffs[8 * i + 6] = (1 << (D - 1)) - r->coeffs[8 * i + 6];
+        r->coeffs[8 * i + 7] = (1 << (D - 1)) - r->coeffs[8 * i + 7];
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_pack
+*
+* Description: Bit-pack polynomial with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYZ_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyz_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[4];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        t[0] = GAMMA1 - a->coeffs[2 * i + 0];
+        t[1] = GAMMA1 - a->coeffs[2 * i + 1];
+
+        r[5 * i + 0]  = t[0];
+        r[5 * i + 1]  = t[0] >> 8;
+        r[5 * i + 2]  = t[0] >> 16;
+        r[5 * i + 2] |= t[1] << 4;
+        r[5 * i + 3]  = t[1] >> 4;
+        r[5 * i + 4]  = t[1] >> 12;
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_unpack
+*
+* Description: Unpack polynomial z with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyz_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        r->coeffs[2 * i + 0]  = a[5 * i + 0];
+        r->coeffs[2 * i + 0] |= (uint32_t)a[5 * i + 1] << 8;
+        r->coeffs[2 * i + 0] |= (uint32_t)a[5 * i + 2] << 16;
+        r->coeffs[2 * i + 0] &= 0xFFFFF;
+
+        r->coeffs[2 * i + 1]  = a[5 * i + 2] >> 4;
+        r->coeffs[2 * i + 1] |= (uint32_t)a[5 * i + 3] << 4;
+        r->coeffs[2 * i + 1] |= (uint32_t)a[5 * i + 4] << 12;
+        r->coeffs[2 * i + 0] &= 0xFFFFF;
+
+        r->coeffs[2 * i + 0] = GAMMA1 - r->coeffs[2 * i + 0];
+        r->coeffs[2 * i + 1] = GAMMA1 - r->coeffs[2 * i + 1];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyw1_pack
+*
+* Description: Bit-pack polynomial w1 with coefficients in [0,15] or [0,43].
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYW1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyw1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        r[i] = a->coeffs[2 * i + 0] | (a->coeffs[2 * i + 1] << 4);
+    }
+
+
+    DBENCH_STOP(*tpack);
+}

src/sign/dilithium/dilithium3/aarch64/poly.h

@@ -0,0 +1,92 @@
+#ifndef POLY_H
+#define POLY_H
+#include "params.h"
+#include <stdint.h>
+
+typedef struct {
+    int32_t coeffs[N];
+} poly;
+
+#define poly_reduce DILITHIUM_NAMESPACE(poly_reduce)
+void poly_reduce(poly *a);
+#define poly_caddq DILITHIUM_NAMESPACE(poly_caddq)
+void poly_caddq(poly *a);
+#define poly_freeze DILITHIUM_NAMESPACE(poly_freeze)
+void poly_freeze(poly *a);
+
+#define poly_add DILITHIUM_NAMESPACE(poly_add)
+void poly_add(poly *c, const poly *a, const poly *b);
+#define poly_sub DILITHIUM_NAMESPACE(poly_sub)
+void poly_sub(poly *c, const poly *a, const poly *b);
+#define poly_shiftl DILITHIUM_NAMESPACE(poly_shiftl)
+void poly_shiftl(poly *a);
+
+#define poly_ntt DILITHIUM_NAMESPACE(poly_ntt)
+void poly_ntt(poly *a);
+#define poly_invntt_tomont DILITHIUM_NAMESPACE(poly_invntt_tomont)
+void poly_invntt_tomont(poly *a);
+#define poly_pointwise_montgomery DILITHIUM_NAMESPACE(poly_pointwise_montgomery)
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b);
+
+#define poly_power2round DILITHIUM_NAMESPACE(poly_power2round)
+void poly_power2round(poly *a1, poly *a0, const poly *a);
+#define poly_decompose DILITHIUM_NAMESPACE(poly_decompose)
+void poly_decompose(poly *a1, poly *a0, const poly *a);
+#define poly_make_hint DILITHIUM_NAMESPACE(poly_make_hint)
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1);
+#define poly_use_hint DILITHIUM_NAMESPACE(poly_use_hint)
+void poly_use_hint(poly *b, const poly *a, const poly *h);
+
+#define poly_chknorm DILITHIUM_NAMESPACE(poly_chknorm)
+int poly_chknorm(const poly *a, int32_t B);
+#define poly_uniform DILITHIUM_NAMESPACE(poly_uniform)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce);
+#define poly_uniformx2 DILITHIUM_NAMESPACE(poly_uniformx2)
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_eta DILITHIUM_NAMESPACE(poly_uniform_eta)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce);
+#define poly_uniform_etax2 DILITHIUM_NAMESPACE(poly_uniform_etax2)
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_gamma1 DILITHIUM_NAMESPACE(poly_uniform_gamma1)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce);
+#define poly_uniform_gamma1x2 DILITHIUM_NAMESPACE(poly_uniform_gamma1x2)
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1);
+#define poly_challenge DILITHIUM_NAMESPACE(poly_challenge)
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define polyeta_pack DILITHIUM_NAMESPACE(polyeta_pack)
+void polyeta_pack(uint8_t *r, const poly *a);
+#define polyeta_unpack DILITHIUM_NAMESPACE(polyeta_unpack)
+void polyeta_unpack(poly *r, const uint8_t *a);
+
+#define polyt1_pack DILITHIUM_NAMESPACE(polyt1_pack)
+void polyt1_pack(uint8_t *r, const poly *a);
+#define polyt1_unpack DILITHIUM_NAMESPACE(polyt1_unpack)
+void polyt1_unpack(poly *r, const uint8_t *a);
+
+#define polyt0_pack DILITHIUM_NAMESPACE(polyt0_pack)
+void polyt0_pack(uint8_t *r, const poly *a);
+#define polyt0_unpack DILITHIUM_NAMESPACE(polyt0_unpack)
+void polyt0_unpack(poly *r, const uint8_t *a);
+
+#define polyz_pack DILITHIUM_NAMESPACE(polyz_pack)
+void polyz_pack(uint8_t *r, const poly *a);
+#define polyz_unpack DILITHIUM_NAMESPACE(polyz_unpack)
+void polyz_unpack(poly *r, const uint8_t *a);
+
+#define polyw1_pack DILITHIUM_NAMESPACE(polyw1_pack)
+void polyw1_pack(uint8_t *r, const poly *a);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/polyvec.c

@@ -0,0 +1,452 @@
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#include "reduce.h"
+
+static const int32_t l_montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+/*************************************************
+* Name:        expand_mat
+*
+* Description: Implementation of ExpandA. Generates matrix A with uniformly
+*              random coefficients a_{i,j} by performing rejection
+*              sampling on the output stream of SHAKE128(rho|j|i)
+*              or AES256CTR(rho,j|i).
+*
+* Arguments:   - polyvecl mat[K]: output matrix
+*              - const uint8_t rho[]: byte array containing seed rho
+**************************************************/
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]) {
+    unsigned int i, j;
+
+    for (j = 0; j < L; ++j) {
+        for (i = 0; i < K; i += 2) {
+            poly_uniformx2(&mat[i + 0].vec[j], &mat[i + 1].vec[j], rho, (uint16_t) ((i << 8) + j), (uint16_t) (((i + 1) << 8) + j));
+        }
+    }
+}
+
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyvecl_pointwise_acc_montgomery(&t->vec[i], &mat[i], v);
+    }
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length L **************/
+/**************************************************************/
+
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+}
+
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L - 1; i += 2) {
+        poly_uniform_gamma1x2(&v->vec[i + 0], &v->vec[i + 1], seed, (uint16_t) (L * nonce + i + 0), (uint16_t) (L * nonce + i + 1));
+    }
+    if (L & 1) {
+        poly_uniform_gamma1(&v->vec[i], seed, (uint16_t) (L * nonce + L - 1));
+    }
+}
+
+void polyvecl_reduce(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length L
+*              to standard representatives.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_freeze(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_add
+*
+* Description: Add vectors of polynomials of length L.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyvecl *w: pointer to output vector
+*              - const polyvecl *u: pointer to first summand
+*              - const polyvecl *v: pointer to second summand
+**************************************************/
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length L. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_ntt(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+void polyvecl_invntt_tomont(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_pointwise_acc_montgomery
+*
+* Description: Pointwise multiply vectors of polynomials of length L, multiply
+*              resulting vector by 2^{-32} and add (accumulate) polynomials
+*              in it. Input/output vectors are in NTT domain representation.
+*
+* Arguments:   - poly *w: output polynomial
+*              - const polyvecl *u: pointer to first input vector
+*              - const polyvecl *v: pointer to second input vector
+**************************************************/
+extern void PQCLEAN_DILITHIUM3_AARCH64_asm_polyvecl_pointwise_acc_montgomery(int32_t *, const int32_t *, const int32_t *, const int32_t *);
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v) {
+    PQCLEAN_DILITHIUM3_AARCH64_asm_polyvecl_pointwise_acc_montgomery(w->coeffs, u->vec[0].coeffs, v->vec[0].coeffs, l_montgomery_const);
+}
+
+/*************************************************
+* Name:        polyvecl_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length L.
+*              Assumes input polyvecl to be reduced by polyvecl_reduce().
+*
+* Arguments:   - const polyvecl *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials is strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyvecl_chknorm(const polyvecl *v, int32_t bound)  {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length K **************/
+/**************************************************************/
+
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+
+}
+
+/*************************************************
+* Name:        polyveck_reduce
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to representatives in [-6283009,6283007].
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_reduce(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_caddq
+*
+* Description: For all coefficients of polynomials in vector of length K
+*              add Q if coefficient is negative.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_caddq(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_caddq(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to standard representatives.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_freeze(polyveck *v)  {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_add
+*
+* Description: Add vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first summand
+*              - const polyveck *v: pointer to second summand
+**************************************************/
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_sub
+*
+* Description: Subtract vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first input vector
+*              - const polyveck *v: pointer to second input vector to be
+*                                   subtracted from first input vector
+**************************************************/
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_sub(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_shiftl
+*
+* Description: Multiply vector of polynomials of Length K by 2^D without modular
+*              reduction. Assumes input coefficients to be less than 2^{31-D}.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_shiftl(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_shiftl(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length K. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_ntt(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_invntt_tomont
+*
+* Description: Inverse NTT and multiplication by 2^{32} of polynomials
+*              in vector of length K. Input coefficients need to be less
+*              than 2*Q.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_invntt_tomont(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+
+/*************************************************
+* Name:        polyveck_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length K.
+*              Assumes input polyveck to be reduced by polyveck_reduce().
+*
+* Arguments:   - const polyveck *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials are strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyveck_chknorm(const polyveck *v, int32_t bound) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        polyveck_power2round
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute a0, a1 such that a mod^+ Q = a1*2^D + a0
+*              with -2^{D-1} < a0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_power2round(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_decompose
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute high and low bits a0, a1 such a mod^+ Q = a1*ALPHA + a0
+*              with -ALPHA/2 < a0 <= ALPHA/2 except a1 = (Q-1)/ALPHA where we
+*              set a1 = 0 and -ALPHA/2 <= a0 = a mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_decompose(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_make_hint
+*
+* Description: Compute hint vector.
+*
+* Arguments:   - polyveck *h: pointer to output vector
+*              - const polyveck *v0: pointer to low part of input vector
+*              - const polyveck *v1: pointer to high part of input vector
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1) {
+    unsigned int i, s = 0;
+
+    for (i = 0; i < K; ++i) {
+        s += poly_make_hint(&h->vec[i], &v0->vec[i], &v1->vec[i]);
+    }
+
+    return s;
+}
+
+/*************************************************
+* Name:        polyveck_use_hint
+*
+* Description: Use hint vector to correct the high bits of input vector.
+*
+* Arguments:   - polyveck *w: pointer to output vector of polynomials with
+*                             corrected high bits
+*              - const polyveck *u: pointer to input vector
+*              - const polyveck *h: pointer to input hint vector
+**************************************************/
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_use_hint(&w->vec[i], &u->vec[i], &h->vec[i]);
+    }
+}
+
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyw1_pack(&r[i * POLYW1_PACKEDBYTES], &w1->vec[i]);
+    }
+}

src/sign/dilithium/dilithium3/aarch64/polyvec.h

@@ -0,0 +1,96 @@
+#ifndef POLYVEC_H
+#define POLYVEC_H
+#include "params.h"
+#include "poly.h"
+#include <stdint.h>
+
+/* Vectors of polynomials of length L */
+typedef struct {
+    poly vec[L];
+} polyvecl;
+
+#define polyvecl_uniform_eta DILITHIUM_NAMESPACE(polyvecl_uniform_eta)
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_uniform_gamma1 DILITHIUM_NAMESPACE(polyvecl_uniform_gamma1)
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_reduce DILITHIUM_NAMESPACE(polyvecl_reduce)
+void polyvecl_reduce(polyvecl *v);
+
+#define polyvecl_freeze DILITHIUM_NAMESPACE(polyvecl_freeze)
+void polyvecl_freeze(polyvecl *v);
+
+#define polyvecl_add DILITHIUM_NAMESPACE(polyvecl_add)
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v);
+
+#define polyvecl_ntt DILITHIUM_NAMESPACE(polyvecl_ntt)
+void polyvecl_ntt(polyvecl *v);
+#define polyvecl_invntt_tomont DILITHIUM_NAMESPACE(polyvecl_invntt_tomont)
+void polyvecl_invntt_tomont(polyvecl *v);
+#define polyvecl_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_poly_montgomery)
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v);
+#define polyvecl_pointwise_acc_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_acc_montgomery)
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v);
+
+
+#define polyvecl_chknorm DILITHIUM_NAMESPACE(polyvecl_chknorm)
+int polyvecl_chknorm(const polyvecl *v, int32_t B);
+
+
+
+/* Vectors of polynomials of length K */
+typedef struct {
+    poly vec[K];
+} polyveck;
+
+#define polyveck_uniform_eta DILITHIUM_NAMESPACE(polyveck_uniform_eta)
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyveck_reduce DILITHIUM_NAMESPACE(polyveck_reduce)
+void polyveck_reduce(polyveck *v);
+#define polyveck_caddq DILITHIUM_NAMESPACE(polyveck_caddq)
+void polyveck_caddq(polyveck *v);
+#define polyveck_freeze DILITHIUM_NAMESPACE(polyveck_freeze)
+void polyveck_freeze(polyveck *v);
+
+#define polyveck_add DILITHIUM_NAMESPACE(polyveck_add)
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_sub DILITHIUM_NAMESPACE(polyveck_sub)
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_shiftl DILITHIUM_NAMESPACE(polyveck_shiftl)
+void polyveck_shiftl(polyveck *v);
+
+#define polyveck_ntt DILITHIUM_NAMESPACE(polyveck_ntt)
+void polyveck_ntt(polyveck *v);
+#define polyveck_invntt_tomont DILITHIUM_NAMESPACE(polyveck_invntt_tomont)
+void polyveck_invntt_tomont(polyveck *v);
+#define polyveck_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyveck_pointwise_poly_montgomery)
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v);
+
+#define polyveck_chknorm DILITHIUM_NAMESPACE(polyveck_chknorm)
+int polyveck_chknorm(const polyveck *v, int32_t B);
+
+#define polyveck_power2round DILITHIUM_NAMESPACE(polyveck_power2round)
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_decompose DILITHIUM_NAMESPACE(polyveck_decompose)
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_make_hint DILITHIUM_NAMESPACE(polyveck_make_hint)
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1);
+#define polyveck_use_hint DILITHIUM_NAMESPACE(polyveck_use_hint)
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h);
+
+#define polyveck_pack_w1 DILITHIUM_NAMESPACE(polyveck_pack_w1)
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1);
+
+#define polyvec_matrix_expand DILITHIUM_NAMESPACE(polyvec_matrix_expand)
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]);
+
+#define polyvec_matrix_pointwise_montgomery DILITHIUM_NAMESPACE(polyvec_matrix_pointwise_montgomery)
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/reduce.c

@@ -0,0 +1,69 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        montgomery_reduce
+*
+* Description: For finite field element a with -2^{31}Q <= a <= Q*2^31,
+*              compute r \equiv a*2^{-32} (mod Q) such that -Q < r < Q.
+*
+* Arguments:   - int64_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t montgomery_reduce(int64_t a) {
+    int32_t t;
+
+    t = (int32_t)((uint64_t)a * (uint64_t)DILITHIUM_QINV);
+    t = (a - (int64_t)t * DILITHIUM_Q) >> 32;
+    return t;
+}
+
+/*************************************************
+* Name:        reduce32
+*
+* Description: For finite field element a with a <= 2^{31} - 2^{22} - 1,
+*              compute r \equiv a (mod Q) such that -6283009 <= r <= 6283007.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t reduce32(int32_t a) {
+    int32_t t;
+
+    t = (a + (1 << 22)) >> 23;
+    t = a - t * DILITHIUM_Q;
+    return t;
+}
+
+/*************************************************
+* Name:        caddq
+*
+* Description: Add Q if input coefficient is negative.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t caddq(int32_t a) {
+    a += (a >> 31) & DILITHIUM_Q;
+    return a;
+}
+
+/*************************************************
+* Name:        freeze
+*
+* Description: For finite field element a, compute standard
+*              representative r = a mod^+ Q.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t freeze(int32_t a) {
+    a = reduce32(a);
+    a = caddq(a);
+    return a;
+}

src/sign/dilithium/dilithium3/aarch64/reduce.h

@@ -0,0 +1,20 @@
+#ifndef REDUCE_H
+#define REDUCE_H
+#include "params.h"
+#include <stdint.h>
+
+#define DILITHIUM_QINV 58728449 // q^(-1) mod 2^32
+
+#define montgomery_reduce DILITHIUM_NAMESPACE(montgomery_reduce)
+int32_t montgomery_reduce(int64_t a);
+
+#define reduce32 DILITHIUM_NAMESPACE(reduce32)
+int32_t reduce32(int32_t a);
+
+#define caddq DILITHIUM_NAMESPACE(caddq)
+int32_t caddq(int32_t a);
+
+#define freeze DILITHIUM_NAMESPACE(freeze)
+int32_t freeze(int32_t a);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/rounding.c

@@ -0,0 +1,96 @@
+#include "params.h"
+#include "rounding.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        power2round
+*
+* Description: For finite field element a, compute a0, a1 such that
+*              a mod^+ Q = a1*2^D + a0 with -2^{D-1} < a0 <= 2^{D-1}.
+*              Assumes a to be standard representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t power2round(int32_t *a0, int32_t a)  {
+    int32_t a1;
+
+    a1 = (a + (1 << (D - 1)) - 1) >> D;
+    *a0 = a - (a1 << D);
+    return a1;
+}
+
+/*************************************************
+* Name:        decompose
+*
+* Description: For finite field element a, compute high and low bits a0, a1 such
+*              that a mod^+ Q = a1*ALPHA + a0 with -ALPHA/2 < a0 <= ALPHA/2 except
+*              if a1 = (Q-1)/ALPHA where we set a1 = 0 and
+*              -ALPHA/2 <= a0 = a mod^+ Q - Q < 0. Assumes a to be standard
+*              representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t decompose(int32_t *a0, int32_t a) {
+    int32_t a1;
+
+    a1  = (a + 127) >> 7;
+
+    a1  = (a1 * 1025 + (1 << 21)) >> 22;
+    a1 &= 15;
+
+
+    *a0  = a - a1 * 2 * GAMMA2;
+    *a0 -= (((DILITHIUM_Q - 1) / 2 - *a0) >> 31) & DILITHIUM_Q;
+    return a1;
+}
+
+/*************************************************
+* Name:        make_hint
+*
+* Description: Compute hint bit indicating whether the low bits of the
+*              input element overflow into the high bits.
+*
+* Arguments:   - int32_t a0: low bits of input element
+*              - int32_t a1: high bits of input element
+*
+* Returns 1 if overflow.
+**************************************************/
+unsigned int make_hint(int32_t a0, int32_t a1) {
+    if (a0 > GAMMA2 || a0 < -GAMMA2 || (a0 == -GAMMA2 && a1 != 0)) {
+        return 1;
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        use_hint
+*
+* Description: Correct high bits according to hint.
+*
+* Arguments:   - int32_t a: input element
+*              - unsigned int hint: hint bit
+*
+* Returns corrected high bits.
+**************************************************/
+int32_t use_hint(int32_t a, unsigned int hint) {
+    int32_t a0, a1;
+
+    a1 = decompose(&a0, a);
+    if (hint == 0) {
+        return a1;
+    }
+
+
+    if (a0 > 0) {
+        return (a1 + 1) & 15;
+    }
+    return (a1 - 1) & 15;
+
+}

src/sign/dilithium/dilithium3/aarch64/rounding.h

@@ -0,0 +1,18 @@
+#ifndef ROUNDING_H
+#define ROUNDING_H
+#include "params.h"
+#include <stdint.h>
+
+#define power2round DILITHIUM_NAMESPACE(power2round)
+int32_t power2round(int32_t *a0, int32_t a);
+
+#define decompose DILITHIUM_NAMESPACE(decompose)
+int32_t decompose(int32_t *a0, int32_t a);
+
+#define make_hint DILITHIUM_NAMESPACE(make_hint)
+unsigned int make_hint(int32_t a0, int32_t a1);
+
+#define use_hint DILITHIUM_NAMESPACE(use_hint)
+int32_t use_hint(int32_t a, unsigned int hint);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/sign.c

@@ -0,0 +1,343 @@
+#include "fips202.h"
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include "randombytes.h"
+#include "sign.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        crypto_sign_keypair
+*
+* Description: Generates public and private key.
+*
+* Arguments:   - uint8_t *pk: pointer to output public key (allocated
+*                             array of CRYPTO_PUBLICKEYBYTES bytes)
+*              - uint8_t *sk: pointer to output private key (allocated
+*                             array of CRYPTO_SECRETKEYBYTES bytes)
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk) {
+    uint8_t seedbuf[2 * SEEDBYTES + CRHBYTES];
+    uint8_t tr[SEEDBYTES];
+    const uint8_t *rho, *rhoprime, *key;
+    polyvecl mat[K];
+    polyvecl s1, s1hat;
+    polyveck s2, t1, t0;
+
+    /* Get randomness for rho, rhoprime and key */
+    randombytes(seedbuf, SEEDBYTES);
+    shake256(seedbuf, 2 * SEEDBYTES + CRHBYTES, seedbuf, SEEDBYTES);
+    rho = seedbuf;
+    rhoprime = rho + SEEDBYTES;
+    key = rhoprime + CRHBYTES;
+
+    /* Expand matrix */
+    polyvec_matrix_expand(mat, rho);
+
+    /* Sample short vectors s1 and s2 */
+    polyvecl_uniform_eta(&s1, rhoprime, 0);
+    polyveck_uniform_eta(&s2, rhoprime, L);
+
+    /* Matrix-vector multiplication */
+    s1hat = s1;
+    polyvecl_ntt(&s1hat);
+    polyvec_matrix_pointwise_montgomery(&t1, mat, &s1hat);
+    polyveck_reduce(&t1);
+    polyveck_invntt_tomont(&t1);
+
+    /* Add error vector s2 */
+    polyveck_add(&t1, &t1, &s2);
+
+    /* Extract t1 and write public key */
+    polyveck_caddq(&t1);
+    polyveck_power2round(&t1, &t0, &t1);
+    pack_pk(pk, rho, &t1);
+
+    /* Compute H(rho, t1) and write secret key */
+    shake256(tr, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    pack_sk(sk, rho, tr, key, &t0, &s1, &s2);
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_signature
+*
+* Description: Computes signature.
+*
+* Arguments:   - uint8_t *sig:   pointer to output signature (of length CRYPTO_BYTES)
+*              - size_t *siglen: pointer to output length of signature
+*              - uint8_t *m:     pointer to message to be signed
+*              - size_t mlen:    length of message
+*              - uint8_t *sk:    pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_signature(uint8_t *sig,
+                          size_t *siglen,
+                          const uint8_t *m,
+                          size_t mlen,
+                          const uint8_t *sk) {
+    unsigned int n;
+    uint8_t seedbuf[3 * SEEDBYTES + 2 * CRHBYTES];
+    uint8_t *rho, *tr, *key, *mu, *rhoprime;
+    uint16_t nonce = 0;
+    polyvecl mat[K], s1, y, z;
+    polyveck t0, s2, w1, w0, h;
+    poly cp;
+    shake256incctx state;
+
+    rho = seedbuf;
+    tr = rho + SEEDBYTES;
+    key = tr + SEEDBYTES;
+    mu = key + SEEDBYTES;
+    rhoprime = mu + CRHBYTES;
+    unpack_sk(rho, tr, key, &t0, &s1, &s2, sk);
+
+    /* Compute CRH(tr, msg) */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, tr, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    shake256(rhoprime, CRHBYTES, key, SEEDBYTES + CRHBYTES);
+
+    /* Expand matrix and transform vectors */
+    polyvec_matrix_expand(mat, rho);
+    polyvecl_ntt(&s1);
+    polyveck_ntt(&s2);
+    polyveck_ntt(&t0);
+
+rej:
+    /* Sample intermediate vector y */
+    polyvecl_uniform_gamma1(&y, rhoprime, nonce++);
+
+    /* Matrix-vector multiplication */
+    z = y;
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Decompose w and call the random oracle */
+    polyveck_caddq(&w1);
+    polyveck_decompose(&w1, &w0, &w1);
+    polyveck_pack_w1(sig, &w1);
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, sig, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(sig, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    poly_challenge(&cp, sig);
+    poly_ntt(&cp);
+
+    /* Compute z, reject if it reveals secret */
+    polyvecl_pointwise_poly_montgomery(&z, &cp, &s1);
+    polyvecl_invntt_tomont(&z);
+    polyvecl_add(&z, &z, &y);
+    polyvecl_reduce(&z);
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        goto rej;
+    }
+
+    /* Check that subtracting cs2 does not change high bits of w and low bits
+     * do not reveal secret information */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &s2);
+    polyveck_invntt_tomont(&h);
+    polyveck_sub(&w0, &w0, &h);
+    polyveck_reduce(&w0);
+    if (polyveck_chknorm(&w0, GAMMA2 - BETA)) {
+        goto rej;
+    }
+
+    /* Compute hints for w1 */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &t0);
+    polyveck_invntt_tomont(&h);
+    polyveck_reduce(&h);
+    if (polyveck_chknorm(&h, GAMMA2)) {
+        goto rej;
+    }
+
+    polyveck_add(&w0, &w0, &h);
+    n = polyveck_make_hint(&h, &w0, &w1);
+    if (n > OMEGA) {
+        goto rej;
+    }
+
+    /* Write signature */
+    pack_sig(sig, sig, &z, &h);
+    *siglen = CRYPTO_BYTES;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign
+*
+* Description: Compute signed message.
+*
+* Arguments:   - uint8_t *sm: pointer to output signed message (allocated
+*                             array with CRYPTO_BYTES + mlen bytes),
+*                             can be equal to m
+*              - size_t *smlen: pointer to output length of signed
+*                               message
+*              - const uint8_t *m: pointer to message to be signed
+*              - size_t mlen: length of message
+*              - const uint8_t *sk: pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign(uint8_t *sm,
+                size_t *smlen,
+                const uint8_t *m,
+                size_t mlen,
+                const uint8_t *sk) {
+    size_t i;
+
+    for (i = 0; i < mlen; ++i) {
+        sm[CRYPTO_BYTES + mlen - 1 - i] = m[mlen - 1 - i];
+    }
+    crypto_sign_signature(sm, smlen, sm + CRYPTO_BYTES, mlen, sk);
+    *smlen += mlen;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_verify
+*
+* Description: Verifies signature.
+*
+* Arguments:   - uint8_t *m: pointer to input signature
+*              - size_t siglen: length of signature
+*              - const uint8_t *m: pointer to message
+*              - size_t mlen: length of message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signature could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_verify(const uint8_t *sig,
+                       size_t siglen,
+                       const uint8_t *m,
+                       size_t mlen,
+                       const uint8_t *pk) {
+    unsigned int i;
+    uint8_t buf[K * POLYW1_PACKEDBYTES];
+    uint8_t rho[SEEDBYTES];
+    uint8_t mu[CRHBYTES];
+    uint8_t c[SEEDBYTES];
+    uint8_t c2[SEEDBYTES];
+    poly cp;
+    polyvecl mat[K], z;
+    polyveck t1, w1, h;
+    shake256incctx state;
+
+    if (siglen != CRYPTO_BYTES) {
+        return -1;
+    }
+
+    unpack_pk(rho, &t1, pk);
+    if (unpack_sig(c, &z, &h, sig)) {
+        return -1;
+    }
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        return -1;
+    }
+
+    /* Compute CRH(H(rho, t1), msg) */
+    shake256(mu, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    /* Matrix-vector multiplication; compute Az - c2^dt1 */
+    poly_challenge(&cp, c);
+    polyvec_matrix_expand(mat, rho);
+
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+
+    poly_ntt(&cp);
+    polyveck_shiftl(&t1);
+    polyveck_ntt(&t1);
+    polyveck_pointwise_poly_montgomery(&t1, &cp, &t1);
+
+    polyveck_sub(&w1, &w1, &t1);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Reconstruct w1 */
+    polyveck_caddq(&w1);
+    polyveck_use_hint(&w1, &w1, &h);
+    polyveck_pack_w1(buf, &w1);
+
+    /* Call random oracle and verify challenge */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, buf, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(c2, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    for (i = 0; i < SEEDBYTES; ++i) {
+        if (c[i] != c2[i]) {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_open
+*
+* Description: Verify signed message.
+*
+* Arguments:   - uint8_t *m: pointer to output message (allocated
+*                            array with smlen bytes), can be equal to sm
+*              - size_t *mlen: pointer to output length of message
+*              - const uint8_t *sm: pointer to signed message
+*              - size_t smlen: length of signed message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signed message could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_open(uint8_t *m,
+                     size_t *mlen,
+                     const uint8_t *sm,
+                     size_t smlen,
+                     const uint8_t *pk) {
+    size_t i;
+
+    if (smlen < CRYPTO_BYTES) {
+        goto badsig;
+    }
+
+    *mlen = smlen - CRYPTO_BYTES;
+    if (crypto_sign_verify(sm, CRYPTO_BYTES, sm + CRYPTO_BYTES, *mlen, pk)) {
+        goto badsig;
+    } else {
+        /* All good, copy msg, return 0 */
+        for (i = 0; i < *mlen; ++i) {
+            m[i] = sm[CRYPTO_BYTES + i];
+        }
+        return 0;
+    }
+
+badsig:
+    /* Signature verification failed */
+    *mlen = (size_t) -1;
+    for (i = 0; i < smlen; ++i) {
+        m[i] = 0;
+    }
+
+    return -1;
+}

src/sign/dilithium/dilithium3/aarch64/sign.h

@@ -0,0 +1,37 @@
+#ifndef SIGN_H
+#define SIGN_H
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stddef.h>
+#include <stdint.h>
+
+
+
+#define challenge DILITHIUM_NAMESPACE(challenge)
+void challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define crypto_sign_keypair DILITHIUM_NAMESPACE(crypto_sign_keypair)
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+#define crypto_sign_signature DILITHIUM_NAMESPACE(crypto_sign_signature)
+int crypto_sign_signature(uint8_t *sig, size_t *siglen,
+                          const uint8_t *m, size_t mlen,
+                          const uint8_t *sk);
+
+#define crypto_sign DILITHIUM_NAMESPACETOP
+int crypto_sign(uint8_t *sm, size_t *smlen,
+                const uint8_t *m, size_t mlen,
+                const uint8_t *sk);
+
+#define crypto_sign_verify DILITHIUM_NAMESPACE(crypto_sign_verify)
+int crypto_sign_verify(const uint8_t *sig, size_t siglen,
+                       const uint8_t *m, size_t mlen,
+                       const uint8_t *pk);
+
+#define crypto_sign_open DILITHIUM_NAMESPACE(crypto_sign_open)
+int crypto_sign_open(uint8_t *m, size_t *mlen,
+                     const uint8_t *sm, size_t smlen,
+                     const uint8_t *pk);
+
+#endif

src/sign/dilithium/dilithium3/aarch64/symmetric-shake.c

@@ -0,0 +1,66 @@
+#include "fips202.h"
+#include "params.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+void dilithium_shake128_stream_init(shake128incctx *state, const uint8_t seed[SEEDBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake128_inc_init(state);
+    shake128_inc_absorb(state, seed, SEEDBYTES);
+    shake128_inc_absorb(state, t, 2);
+    shake128_inc_finalize(state);
+}
+
+void dilithium_shake256_stream_init(shake256incctx *state, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake256_inc_init(state);
+    shake256_inc_absorb(state, seed, CRHBYTES);
+    shake256_inc_absorb(state, t, 2);
+    shake256_inc_finalize(state);
+}
+
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[SEEDBYTES + 2 + 14];
+    uint8_t extseed2[SEEDBYTES + 2 + 14];
+
+    for (i = 0; i < SEEDBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[SEEDBYTES] = (uint8_t) nonce1;
+    extseed1[SEEDBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[SEEDBYTES  ] = (uint8_t) nonce2;
+    extseed2[SEEDBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake128x2_absorb(state, extseed1, extseed2, SEEDBYTES + 2);
+}
+
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[CRHBYTES + 2 + 14];
+    uint8_t extseed2[CRHBYTES + 2 + 14];
+
+    for (i = 0; i < CRHBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[CRHBYTES] = (uint8_t) nonce1;
+    extseed1[CRHBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[CRHBYTES  ] = (uint8_t) nonce2;
+    extseed2[CRHBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake256x2_absorb(state, extseed1, extseed2, CRHBYTES + 2);
+}

src/sign/dilithium/dilithium3/aarch64/symmetric.h

@@ -0,0 +1,47 @@
+#ifndef SYMMETRIC_H
+#define SYMMETRIC_H
+#include "fips202.h"
+#include "fips202x2.h"
+#include "params.h"
+#include <stdint.h>
+
+
+typedef shake128incctx stream128_state;
+typedef shake256incctx stream256_state;
+
+#define dilithium_shake128_stream_init DILITHIUM_NAMESPACE(dilithium_shake128_stream_init)
+void dilithium_shake128_stream_init(shake128incctx *state,
+                                    const uint8_t seed[SEEDBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake256_stream_init DILITHIUM_NAMESPACE(dilithium_shake256_stream_init)
+void dilithium_shake256_stream_init(shake256incctx *state,
+                                    const uint8_t seed[CRHBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake128x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake128x2_stream_init)
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+#define dilithium_shake256x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake256x2_stream_init)
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+
+
+#define STREAM128_BLOCKBYTES SHAKE128_RATE
+#define STREAM256_BLOCKBYTES SHAKE256_RATE
+
+#define stream128_init(STATE, SEED, NONCE) \
+    dilithium_shake128_stream_init(STATE, SEED, NONCE)
+#define stream128_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake128_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE128_RATE), STATE)
+#define stream128_release(STATE) shake128_inc_ctx_release(STATE)
+#define stream256_init(STATE, SEED, NONCE) \
+    dilithium_shake256_stream_init(STATE, SEED, NONCE)
+#define stream256_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake256_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE256_RATE), STATE)
+#define stream256_release(STATE) shake256_inc_ctx_release(STATE)
+
+
+#endif

src/sign/dilithium/dilithium5/aarch64/LICENSE

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.

src/sign/dilithium/dilithium5/aarch64/NTT_params.h

@@ -0,0 +1,56 @@
+#ifndef NTT_PARAMS_H
+#define NTT_PARAMS_H
+
+#define ARRAY_N 256
+
+#define NTT_N 256
+#define LOGNTT_N 8
+
+// root of unity: 1753
+
+
+// Q1
+#define Q1 8380417
+// omegaQ1 = 1753 mod Q1
+#define omegaQ1 1753
+// invomegaQ1 = omegaQ^{-1} mod Q1
+#define invomegaQ1 731434
+// R = 2^32 below
+// RmodQ1 = 2^32 mod^{+-} Q1
+#define RmodQ1 (-4186625)
+// Q1prime = Q1^{-1} mod^{+-} 2^32
+#define Q1prime 58728449
+// invNQ1 = NTT_N^{-1} mod Q1
+#define invNQ1 8347681
+
+// invNQ1R2modQ1 = -NTT_N^{-1} 2^32 2^32 mod^{+-} Q1 below
+#define invNQ1R2modQ1 (-41978)
+// invNQ1R2modQ1_prime = invNQ1R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime 8395782
+// invNQ1R2modQ1_prime_half = (invNQ1R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1R2modQ1_prime_half 4197891
+// invNQ1R2modQ1_doubleprime = (invNQ1R2modQ1_prime Q1 - invNQ1R2modQ1) / 2^32
+#define invNQ1R2modQ1_doubleprime 16382
+
+// invNQ1_final_R2modQ1 = -invNQ1R2modQ1 invomegaQ1^{128} mod q
+#define invNQ1_final_R2modQ1 4404704
+// invNQ1_final_R2modQ1_prime = invNQ1_final_R2modQ1 (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime (-151046688)
+// invNQ1_final_R2modQ1_prime_half = (invNQ1_final_R2modQ1 / 2) (Q1^{-1} mod^{+-} 2^32) mod^{+-} 2^32
+#define invNQ1_final_R2modQ1_prime_half (-75523344)
+// invNQ1_final_R2modQ1_doubleprime = (invNQ1_final_R2modQ1_prime Q1 - invNQ1_final_R2modQ1) / 2^32
+#define invNQ1_final_R2modQ1_doubleprime (-294725)
+
+// RmodQ1_prime = -(RmodQ1 + Q1) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime 512
+// RmodQ1_prime_half = ( -(RmodQ1 + Q1) / 2) Q1prime mod^{+-} 2^32
+#define RmodQ1_prime_half 256
+// RmodQ1_doubleprime = (RmodQ1_prime Q1 - RmodQ1_prime ) / 2^32
+#define RmodQ1_doubleprime 1
+
+#endif
+
+
+
+
+

src/sign/dilithium/dilithium5/aarch64/__asm_NTT.S

@@ -0,0 +1,274 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_top
+.global _PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_top:
+_PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    src1      .req x1
+    src2      .req x2
+    src3      .req x3
+    src4      .req x4
+    src5      .req x5
+    src6      .req x6
+    src7      .req x7
+    src8      .req x8
+    src9      .req x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v1.4S}, [ src1]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v7.4S}, [ src7]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v11.4S}, [src11]
+    ld1 {v13.4S}, [src13]
+    ld1 {v15.4S}, [src15]
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v8.4S}, [ src8]
+    ld1 {v10.4S}, [src10]
+    ld1 {v12.4S}, [src12]
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    mov counter, #3
+    _ntt_top_loop:
+
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+
+    qq_butterfly_top  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v5,  v7,  v9, v11, v13, v15, v16, v17, v18, v19,  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v8, v10, v12, v14, v28, v29, v30, v31,  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v1,  v3,  v9, v11,  v5,  v7, v13, v15, v16, v17, v18, v19,  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v8, v10,  v4,  v6, v12, v14, v28, v29, v30, v31,  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19, v20, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v1,  v5,  v9, v13,  v3,  v7, v11, v15, v16, v17, v18, v19,  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3
+    qq_butterfly_mixed  v0,  v4,  v8, v12,  v2,  v6, v10, v14, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19, v20, v22, 0, 1, v22, 2, 3, v23, 0, 1, v23, 2, 3, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed  v0,  v2,  v4,  v6,  v1,  v3,  v5,  v7, v16, v17, v18, v19,  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20,  v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_bot  v8, v10, v12, v14,  v9, v11, v13, v15, v28, v29, v30, v31, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_top_loop
+
+    st1 { v1.4S}, [ src1], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v7.4S}, [ src7], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v15.4S}, [src15], #16
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v8.4S}, [ src8], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v14.4S}, [src14], #16
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_bot
+.global _PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_bot:
+_PQCLEAN_DILITHIUM5_AARCH64_asm_ntt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _ntt_bot_loop:
+
+    ld1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld1 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+
+    dq_butterfly_top  v0,  v1,  v2,  v3, v12, v13, v4,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v20, 2, 3, v20, 2, 3,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_bot v16, v18, v17, v19, v28, v29, v4, v21, 0, 1, v21, 2, 3
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_vec_top  v0,  v1,  v2,  v3, v12, v13, v4,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed  v0,  v1,  v2,  v3, v12, v13, v16, v17, v18, v19, v28, v29, v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_mixed v16, v17, v18, v19, v28, v29,  v0,  v2,  v1,  v3, v12, v13, v4, v22, v23, v22, v23,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed  v0,  v2,  v1,  v3, v12, v13, v16, v18, v17, v19, v28, v29, v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_bot v16, v18, v17, v19, v28, v29, v4, v24, v25, v26, v27
+
+    st4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st4 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _ntt_bot_loop
+
+    .unreq    Q
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+
+
+
+
+
+
+
+
+

src/sign/dilithium/dilithium5/aarch64/__asm_iNTT.S

@@ -0,0 +1,509 @@
+
+#include "macros.inc"
+
+.align 2
+.global PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_top
+.global _PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_top
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_top, %function
+#endif
+PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_top:
+_PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_top:
+
+    push_all
+    Q         .req w20
+    Qhalf     .req w21
+    nQhalf    .req w22
+    invNR2ph  .req w24
+    invNR2dp  .req w25
+    invNWR2ph .req w26
+    invNWR2dp .req w27
+    src0      .req  x0
+    src1      .req  x1
+    src2      .req  x2
+    src3      .req  x3
+    src4      .req  x4
+    src5      .req  x5
+    src6      .req  x6
+    src7      .req  x7
+    src8      .req  x8
+    src9      .req  x9
+    src10     .req x10
+    src11     .req x11
+    src12     .req x12
+    src13     .req x13
+    src14     .req x14
+    src15     .req x15
+    table     .req x28
+    counter   .req x19
+
+    ldr Q, [x2, #0]
+    lsr Qhalf, Q, #1
+    neg nQhalf, Qhalf
+
+    ldr invNR2ph,  [x2, #16]
+    ldr invNR2dp,  [x2, #20]
+    ldr invNWR2ph, [x2, #24]
+    ldr invNWR2dp, [x2, #28]
+
+    mov table, x1
+
+    add  src1, src0, #64
+    add  src2, src0, #128
+
+    add  src3, src0, #192
+    add  src4, src0, #256
+
+    add  src5, src0, #320
+    add  src6, src0, #384
+
+    add  src7, src0, #448
+    add  src8, src0, #512
+
+    add  src9, src0, #576
+    add src10, src0, #640
+
+    add src11, src0, #704
+    add src12, src0, #768
+
+    add src13, src0, #832
+    add src14, src0, #896
+
+    add src15, src0, #960
+
+    ld1 {v20.4S, v21.4S, v22.4S, v23.4S}, [table], #64
+    ld1 {v24.4S, v25.4S, v26.4S, v27.4S}, [table], #64
+
+    mov v20.S[0], Q
+
+    ld1 { v0.4S}, [ src0]
+    ld1 { v1.4S}, [ src1]
+    ld1 { v2.4S}, [ src2]
+    ld1 { v3.4S}, [ src3]
+    ld1 { v4.4S}, [ src4]
+    ld1 { v5.4S}, [ src5]
+    ld1 { v6.4S}, [ src6]
+    ld1 { v7.4S}, [ src7]
+
+    ld1 { v8.4S}, [ src8]
+    ld1 { v9.4S}, [ src9]
+    ld1 {v10.4S}, [src10]
+    ld1 {v11.4S}, [src11]
+    ld1 {v12.4S}, [src12]
+    ld1 {v13.4S}, [src13]
+    ld1 {v14.4S}, [src14]
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    mov counter, #3
+    _intt_top_loop:
+
+    st1 { v0.4S}, [ src0], #16
+    ld1 { v0.4S}, [ src0]
+    st1 { v1.4S}, [ src1], #16
+    ld1 { v1.4S}, [ src1]
+    st1 { v2.4S}, [ src2], #16
+    ld1 { v2.4S}, [ src2]
+    st1 { v3.4S}, [ src3], #16
+    ld1 { v3.4S}, [ src3]
+    st1 { v4.4S}, [ src4], #16
+    ld1 { v4.4S}, [ src4]
+    st1 { v5.4S}, [ src5], #16
+    ld1 { v5.4S}, [ src5]
+    st1 { v6.4S}, [ src6], #16
+    ld1 { v6.4S}, [ src6]
+    st1 { v7.4S}, [ src7], #16
+    ld1 { v7.4S}, [ src7]
+
+    st1 { v8.4S}, [ src8], #16
+    ld1 { v8.4S}, [ src8]
+    st1 { v9.4S}, [ src9], #16
+    ld1 { v9.4S}, [ src9]
+    st1 {v10.4S}, [src10], #16
+    ld1 {v10.4S}, [src10]
+    st1 {v11.4S}, [src11], #16
+    ld1 {v11.4S}, [src11]
+    st1 {v12.4S}, [src12], #16
+    ld1 {v12.4S}, [src12]
+    st1 {v13.4S}, [src13], #16
+    ld1 {v13.4S}, [src13]
+    st1 {v14.4S}, [src14], #16
+    ld1 {v14.4S}, [src14]
+    st1 {v15.4S}, [src15], #16
+    ld1 {v15.4S}, [src15]
+
+    qq_butterfly_bot  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3
+    qq_butterfly_mixed_rev  v0,  v2,  v4,  v6, v16, v17, v18, v19,  v1,  v3,  v5,  v7,  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15, v20, v24, 0, 1, v24, 2, 3, v25, 0, 1, v25, 2, 3, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3
+    qq_butterfly_mixed_rev  v8, v10, v12, v14, v28, v29, v30, v31,  v9, v11, v13, v15,  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7, v20, v26, 0, 1, v26, 2, 3, v27, 0, 1, v27, 2, 3, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3
+    qq_butterfly_mixed_rev  v0,  v1,  v4,  v5, v16, v17, v18, v19,  v2,  v3,  v6,  v7,  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15, v20, v22, 0, 1, v22, 0, 1, v22, 2, 3, v22, 2, 3, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3
+    qq_butterfly_mixed_rev  v8,  v9, v12, v13, v28, v29, v30, v31, v10, v11, v14, v15,  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7, v20, v23, 0, 1, v23, 0, 1, v23, 2, 3, v23, 2, 3, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1
+    qq_butterfly_mixed_rev  v0,  v1,  v2,  v3, v16, v17, v18, v19,  v4,  v5,  v6,  v7,  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 0, 1, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+    qq_butterfly_top  v8,  v9, v10, v11, v28, v29, v30, v31, v12, v13, v14, v15, v20, v21, 2, 3, v21, 2, 3, v21, 2, 3, v21, 2, 3
+
+    mov v20.S[2], invNWR2ph
+    mov v20.S[3], invNWR2dp
+
+    qq_sub_add v16, v17, v18, v19, v28, v29, v30, v31,  v0,  v2,  v4,  v6,  v8, v10, v12, v14
+    qq_sub_add  v0,  v2,  v4,  v6,  v8, v10, v12, v14,  v1,  v3,  v5,  v7,  v9, v11, v13, v15
+
+    qq_montgomery_mul  v9, v11, v13, v15,  v8, v10, v12, v14, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v8, v10, v12, v14, v28, v29, v30, v31, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    mov v20.S[2], invNR2ph
+    mov v20.S[3], invNR2dp
+
+    qq_montgomery_mul  v1,  v3,  v5,  v7,  v0,  v2,  v4,  v6, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+    qq_montgomery_mul  v0,  v2,  v4,  v6, v16, v17, v18, v19, v20, v20, 2, 3, v20, 2, 3, v20, 2, 3, v20, 2, 3
+
+    dup v29.4S, Q
+    dup v30.4S, Qhalf
+    dup v31.4S, nQhalf
+
+    cmge v18.4S, v31.4S,  v0.4S
+    cmge v19.4S, v31.4S,  v1.4S
+    cmge v16.4S,  v0.4S, v30.4S
+    cmge v17.4S,  v1.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v0.4S, v16.4S, v29.4S
+    mla   v1.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v2.4S
+    cmge v19.4S, v31.4S,  v3.4S
+    cmge v16.4S,  v2.4S, v30.4S
+    cmge v17.4S,  v3.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v2.4S, v16.4S, v29.4S
+    mla   v3.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v4.4S
+    cmge v19.4S, v31.4S,  v5.4S
+    cmge v16.4S,  v4.4S, v30.4S
+    cmge v17.4S,  v5.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v4.4S, v16.4S, v29.4S
+    mla   v5.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v6.4S
+    cmge v19.4S, v31.4S,  v7.4S
+    cmge v16.4S,  v6.4S, v30.4S
+    cmge v17.4S,  v7.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v6.4S, v16.4S, v29.4S
+    mla   v7.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S,  v8.4S
+    cmge v19.4S, v31.4S,  v9.4S
+    cmge v16.4S,  v8.4S, v30.4S
+    cmge v17.4S,  v9.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla   v8.4S, v16.4S, v29.4S
+    mla   v9.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v10.4S
+    cmge v19.4S, v31.4S, v11.4S
+    cmge v16.4S, v10.4S, v30.4S
+    cmge v17.4S, v11.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v10.4S, v16.4S, v29.4S
+    mla  v11.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v12.4S
+    cmge v19.4S, v31.4S, v13.4S
+    cmge v16.4S, v12.4S, v30.4S
+    cmge v17.4S, v13.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v12.4S, v16.4S, v29.4S
+    mla  v13.4S, v17.4S, v29.4S
+
+    cmge v18.4S, v31.4S, v14.4S
+    cmge v19.4S, v31.4S, v15.4S
+    cmge v16.4S, v14.4S, v30.4S
+    cmge v17.4S, v15.4S, v30.4S
+
+    sub  v16.4S, v16.4S, v18.4S
+    sub  v17.4S, v17.4S, v19.4S
+
+    mla  v14.4S, v16.4S, v29.4S
+    mla  v15.4S, v17.4S, v29.4S
+
+    sub counter, counter, #1
+    cbnz counter, _intt_top_loop
+
+    st1 { v0.4S}, [ src0], #16
+    st1 { v1.4S}, [ src1], #16
+    st1 { v2.4S}, [ src2], #16
+    st1 { v3.4S}, [ src3], #16
+    st1 { v4.4S}, [ src4], #16
+    st1 { v5.4S}, [ src5], #16
+    st1 { v6.4S}, [ src6], #16
+    st1 { v7.4S}, [ src7], #16
+
+    st1 { v8.4S}, [ src8], #16
+    st1 { v9.4S}, [ src9], #16
+    st1 {v10.4S}, [src10], #16
+    st1 {v11.4S}, [src11], #16
+    st1 {v12.4S}, [src12], #16
+    st1 {v13.4S}, [src13], #16
+    st1 {v14.4S}, [src14], #16
+    st1 {v15.4S}, [src15], #16
+
+    .unreq    Q
+    .unreq    Qhalf
+    .unreq    nQhalf
+    .unreq    invNR2ph
+    .unreq    invNR2dp
+    .unreq    invNWR2ph
+    .unreq    invNWR2dp
+    .unreq    src0
+    .unreq    src1
+    .unreq    src2
+    .unreq    src3
+    .unreq    src4
+    .unreq    src5
+    .unreq    src6
+    .unreq    src7
+    .unreq    src8
+    .unreq    src9
+    .unreq    src10
+    .unreq    src11
+    .unreq    src12
+    .unreq    src13
+    .unreq    src14
+    .unreq    src15
+    .unreq    table
+    .unreq    counter
+    pop_all
+
+    br lr
+
+.align 2
+.global PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_bot
+.global _PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_bot
+#ifndef __clang__
+.type PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_bot, %function
+#endif
+PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_bot:
+_PQCLEAN_DILITHIUM5_AARCH64_asm_intt_SIMD_bot:
+
+    push_all
+    Q         .req w20
+    RphRdp    .req x21
+    src0      .req x0
+    des0      .req x1
+    src1      .req x2
+    des1      .req x3
+    table0    .req x28
+    table1    .req x27
+    counter   .req x19
+
+    ldr Q, [x2]
+    ldr RphRdp, [x2, #8]
+
+    add table0, x1, #128
+    add table1, table0, #1024
+
+    add src1, src0, #512    
+
+    add des0, src0, #0
+    add des1, src0, #512
+
+    mov counter, #8
+    _intt_bot_loop:
+
+    ld4 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [src0], #64
+    ld4 { v16.4S, v17.4S, v18.4S, v19.4S}, [src1], #64
+
+    ld1 {  v4.4S,  v5.4S}, [table0], #32
+    ld2 {  v6.4S,  v7.4S}, [table0], #32
+    ld4 {  v8.4S,  v9.4S, v10.4S, v11.4S}, [table0], #64
+    ld1 { v20.4S, v21.4S}, [table1], #32
+    ld2 { v22.4S, v23.4S}, [table1], #32
+    ld4 { v24.4S, v25.4S, v26.4S, v27.4S}, [table1], #64
+
+    mov v4.S[0], Q
+    mov v20.D[0], RphRdp
+
+    dq_butterfly_vec_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v8,  v9, v10, v11
+    dq_butterfly_vec_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v8,  v9, v10, v11, v24, v25, v26, v27
+    dq_butterfly_vec_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v24, v25, v26, v27,  v6,  v7,  v6,  v7
+    dq_butterfly_vec_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v6,  v7,  v6,  v7, v22, v23, v22, v23
+    dq_butterfly_vec_top v16, v17, v28, v29, v18, v19,  v4, v22, v23, v22, v23
+
+    trn_4x4  v0,  v1,  v2,  v3, v12, v13, v14, v15
+    trn_4x4 v16, v17, v18, v19, v28, v29, v30, v31
+
+    dq_butterfly_bot  v0,  v2, v12, v13,  v1,  v3,  v4,  v5, 0, 1,  v5, 2, 3
+    dq_butterfly_mixed_rev  v0,  v2, v12, v13,  v1,  v3, v16, v18, v28, v29, v17, v19,  v4,  v5, 0, 1,  v5, 2, 3, v21, 0, 1, v21, 2, 3
+    dq_butterfly_mixed_rev v16, v18, v28, v29, v17, v19,  v0,  v1, v12, v13,  v2,  v3,  v4, v21, 0, 1, v21, 2, 3,  v4, 2, 3,  v4, 2, 3
+    dq_butterfly_mixed_rev  v0,  v1, v12, v13,  v2,  v3, v16, v17, v28, v29, v18, v19,  v4,  v4, 2, 3,  v4, 2, 3, v20, 2, 3, v20, 2, 3
+    dq_butterfly_top v16, v17, v28, v29, v18, v19,  v4, v20, 2, 3, v20, 2, 3
+
+    srshr v14.4S,  v0.4S, #23
+    srshr v15.4S,  v1.4S, #23
+    srshr v30.4S, v16.4S, #23
+    srshr v31.4S, v17.4S, #23
+
+    mls    v0.4S, v14.4S, v4.S[0]
+    mls    v1.4S, v15.4S, v4.S[0]
+    mls   v16.4S, v30.4S, v4.S[0]
+    mls   v17.4S, v31.4S, v4.S[0]
+
+    st1 {  v0.4S,  v1.4S,  v2.4S,  v3.4S}, [des0], #64
+    st1 { v16.4S, v17.4S, v18.4S, v19.4S}, [des1], #64
+
+    sub counter, counter, #1
+    cbnz counter, _intt_bot_loop
+
+    .unreq    Q
+    .unreq    RphRdp
+    .unreq    src0
+    .unreq    des0
+    .unreq    src1
+    .unreq    des1
+    .unreq    table0
+    .unreq    table1
+    .unreq    counter
+    pop_all
+
+    br lr
+
+
+
+
+
+

src/sign/dilithium/dilithium5/aarch64/api.h

@@ -0,0 +1,33 @@
+#ifndef PQCLEAN_DILITHIUM5_AARCH64_API_H
+#define PQCLEAN_DILITHIUM5_AARCH64_API_H
+
+
+#include <stddef.h>
+#include <stdint.h>
+
+#define PQCLEAN_DILITHIUM5_AARCH64_CRYPTO_PUBLICKEYBYTES 2592
+#define PQCLEAN_DILITHIUM5_AARCH64_CRYPTO_SECRETKEYBYTES 4864
+#define PQCLEAN_DILITHIUM5_AARCH64_CRYPTO_BYTES 4595
+#define PQCLEAN_DILITHIUM5_AARCH64_CRYPTO_ALGNAME "Dilithium5"
+
+
+int PQCLEAN_DILITHIUM5_AARCH64_crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+int PQCLEAN_DILITHIUM5_AARCH64_crypto_sign_signature(
+    uint8_t *sig, size_t *siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM5_AARCH64_crypto_sign_verify(
+    const uint8_t *sig, size_t siglen,
+    const uint8_t *m, size_t mlen, const uint8_t *pk);
+
+int PQCLEAN_DILITHIUM5_AARCH64_crypto_sign(
+    uint8_t *sm, size_t *smlen,
+    const uint8_t *m, size_t mlen, const uint8_t *sk);
+
+int PQCLEAN_DILITHIUM5_AARCH64_crypto_sign_open(
+    uint8_t *m, size_t *mlen,
+    const uint8_t *sm, size_t smlen, const uint8_t *pk);
+
+
+#endif

src/sign/dilithium/dilithium5/aarch64/fips202x2.c

@@ -0,0 +1,646 @@
+#include "fips202x2.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+
+#define NROUNDS 24
+
+// Define NEON operation
+// c = load(ptr)
+#define vload(ptr) vld1q_u64(ptr);
+// ptr <= c;
+#define vstore(ptr, c) vst1q_u64(ptr, c);
+// c = a ^ b
+#define vxor(c, a, b) c = veorq_u64(a, b);
+// Rotate by n bit ((a << offset) ^ (a >> (64-offset)))
+#define vROL(out, a, offset)    \
+    (out) = vshlq_n_u64(a, offset); \
+    (out) = vsriq_n_u64(out, a, 64 - (offset));
+// Xor chain: out = a ^ b ^ c ^ d ^ e
+#define vXOR4(out, a, b, c, d, e) \
+    (out) = veorq_u64(a, b);          \
+    (out) = veorq_u64(out, c);        \
+    (out) = veorq_u64(out, d);        \
+    (out) = veorq_u64(out, e);
+// Not And c = ~a & b
+// #define vbic(c, a, b) c = vbicq_u64(b, a);
+// Xor Not And: out = a ^ ( (~b) & c)
+#define vXNA(out, a, b, c) \
+    (out) = vbicq_u64(c, b);   \
+    (out) = veorq_u64(out, a);
+// Rotate by 1 bit, then XOR: a ^ ROL(b): SHA1 instruction, not support
+#define vrxor(c, a, b) c = vrax1q_u64(a, b);
+// End Define
+
+/* Keccak round constants */
+static const uint64_t neon_KeccakF_RoundConstants[NROUNDS] = {
+    (uint64_t)0x0000000000000001ULL,
+    (uint64_t)0x0000000000008082ULL,
+    (uint64_t)0x800000000000808aULL,
+    (uint64_t)0x8000000080008000ULL,
+    (uint64_t)0x000000000000808bULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008009ULL,
+    (uint64_t)0x000000000000008aULL,
+    (uint64_t)0x0000000000000088ULL,
+    (uint64_t)0x0000000080008009ULL,
+    (uint64_t)0x000000008000000aULL,
+    (uint64_t)0x000000008000808bULL,
+    (uint64_t)0x800000000000008bULL,
+    (uint64_t)0x8000000000008089ULL,
+    (uint64_t)0x8000000000008003ULL,
+    (uint64_t)0x8000000000008002ULL,
+    (uint64_t)0x8000000000000080ULL,
+    (uint64_t)0x000000000000800aULL,
+    (uint64_t)0x800000008000000aULL,
+    (uint64_t)0x8000000080008081ULL,
+    (uint64_t)0x8000000000008080ULL,
+    (uint64_t)0x0000000080000001ULL,
+    (uint64_t)0x8000000080008008ULL
+};
+
+/*************************************************
+* Name:        KeccakF1600_StatePermutex2
+*
+* Description: The Keccak F1600 Permutation
+*
+* Arguments:   - uint64_t *state: pointer to input/output Keccak state
+**************************************************/
+static inline
+void KeccakF1600_StatePermutex2(v128 state[25]) {
+    v128 Aba, Abe, Abi, Abo, Abu;
+    v128 Aga, Age, Agi, Ago, Agu;
+    v128 Aka, Ake, Aki, Ako, Aku;
+    v128 Ama, Ame, Ami, Amo, Amu;
+    v128 Asa, Ase, Asi, Aso, Asu;
+    v128 BCa, BCe, BCi, BCo, BCu; // tmp
+    v128 Da, De, Di, Do, Du;      // D
+    v128 Eba, Ebe, Ebi, Ebo, Ebu;
+    v128 Ega, Ege, Egi, Ego, Egu;
+    v128 Eka, Eke, Eki, Eko, Eku;
+    v128 Ema, Eme, Emi, Emo, Emu;
+    v128 Esa, Ese, Esi, Eso, Esu;
+
+    //copyFromState(A, state)
+    Aba = state[0];
+    Abe = state[1];
+    Abi = state[2];
+    Abo = state[3];
+    Abu = state[4];
+    Aga = state[5];
+    Age = state[6];
+    Agi = state[7];
+    Ago = state[8];
+    Agu = state[9];
+    Aka = state[10];
+    Ake = state[11];
+    Aki = state[12];
+    Ako = state[13];
+    Aku = state[14];
+    Ama = state[15];
+    Ame = state[16];
+    Ami = state[17];
+    Amo = state[18];
+    Amu = state[19];
+    Asa = state[20];
+    Ase = state[21];
+    Asi = state[22];
+    Aso = state[23];
+    Asu = state[24];
+
+    for (int round = 0; round < NROUNDS; round += 2) {
+        //    prepareTheta
+        vXOR4(BCa, Aba, Aga, Aka, Ama, Asa);
+        vXOR4(BCe, Abe, Age, Ake, Ame, Ase);
+        vXOR4(BCi, Abi, Agi, Aki, Ami, Asi);
+        vXOR4(BCo, Abo, Ago, Ako, Amo, Aso);
+        vXOR4(BCu, Abu, Agu, Aku, Amu, Asu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round  , A, E)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Aba, Aba, Da);
+        vxor(Age, Age, De);
+        vROL(BCe, Age, 44);
+        vxor(Aki, Aki, Di);
+        vROL(BCi, Aki, 43);
+        vxor(Amo, Amo, Do);
+        vROL(BCo, Amo, 21);
+        vxor(Asu, Asu, Du);
+        vROL(BCu, Asu, 14);
+        vXNA(Eba, Aba, BCe, BCi);
+        vxor(Eba, Eba, vdupq_n_u64(neon_KeccakF_RoundConstants[round]));
+        vXNA(Ebe, BCe, BCi, BCo);
+        vXNA(Ebi, BCi, BCo, BCu);
+        vXNA(Ebo, BCo, BCu, Aba);
+        vXNA(Ebu, BCu, Aba, BCe);
+
+        vxor(Abo, Abo, Do);
+        vROL(BCa, Abo, 28);
+        vxor(Agu, Agu, Du);
+        vROL(BCe, Agu, 20);
+        vxor(Aka, Aka, Da);
+        vROL(BCi, Aka, 3);
+        vxor(Ame, Ame, De);
+        vROL(BCo, Ame, 45);
+        vxor(Asi, Asi, Di);
+        vROL(BCu, Asi, 61);
+        vXNA(Ega, BCa, BCe, BCi);
+        vXNA(Ege, BCe, BCi, BCo);
+        vXNA(Egi, BCi, BCo, BCu);
+        vXNA(Ego, BCo, BCu, BCa);
+        vXNA(Egu, BCu, BCa, BCe);
+
+        vxor(Abe, Abe, De);
+        vROL(BCa, Abe, 1);
+        vxor(Agi, Agi, Di);
+        vROL(BCe, Agi, 6);
+        vxor(Ako, Ako, Do);
+        vROL(BCi, Ako, 25);
+        vxor(Amu, Amu, Du);
+        vROL(BCo, Amu, 8);
+        vxor(Asa, Asa, Da);
+        vROL(BCu, Asa, 18);
+        vXNA(Eka, BCa, BCe, BCi);
+        vXNA(Eke, BCe, BCi, BCo);
+        vXNA(Eki, BCi, BCo, BCu);
+        vXNA(Eko, BCo, BCu, BCa);
+        vXNA(Eku, BCu, BCa, BCe);
+
+        vxor(Abu, Abu, Du);
+        vROL(BCa, Abu, 27);
+        vxor(Aga, Aga, Da);
+        vROL(BCe, Aga, 36);
+        vxor(Ake, Ake, De);
+        vROL(BCi, Ake, 10);
+        vxor(Ami, Ami, Di);
+        vROL(BCo, Ami, 15);
+        vxor(Aso, Aso, Do);
+        vROL(BCu, Aso, 56);
+        vXNA(Ema, BCa, BCe, BCi);
+        vXNA(Eme, BCe, BCi, BCo);
+        vXNA(Emi, BCi, BCo, BCu);
+        vXNA(Emo, BCo, BCu, BCa);
+        vXNA(Emu, BCu, BCa, BCe);
+
+        vxor(Abi, Abi, Di);
+        vROL(BCa, Abi, 62);
+        vxor(Ago, Ago, Do);
+        vROL(BCe, Ago, 55);
+        vxor(Aku, Aku, Du);
+        vROL(BCi, Aku, 39);
+        vxor(Ama, Ama, Da);
+        vROL(BCo, Ama, 41);
+        vxor(Ase, Ase, De);
+        vROL(BCu, Ase, 2);
+        vXNA(Esa, BCa, BCe, BCi);
+        vXNA(Ese, BCe, BCi, BCo);
+        vXNA(Esi, BCi, BCo, BCu);
+        vXNA(Eso, BCo, BCu, BCa);
+        vXNA(Esu, BCu, BCa, BCe);
+
+        // Next Round
+
+        //    prepareTheta
+        vXOR4(BCa, Eba, Ega, Eka, Ema, Esa);
+        vXOR4(BCe, Ebe, Ege, Eke, Eme, Ese);
+        vXOR4(BCi, Ebi, Egi, Eki, Emi, Esi);
+        vXOR4(BCo, Ebo, Ego, Eko, Emo, Eso);
+        vXOR4(BCu, Ebu, Egu, Eku, Emu, Esu);
+
+        //thetaRhoPiChiIotaPrepareTheta(round+1, E, A)
+        vROL(Da, BCe, 1);
+        vxor(Da, BCu, Da);
+        vROL(De, BCi, 1);
+        vxor(De, BCa, De);
+        vROL(Di, BCo, 1);
+        vxor(Di, BCe, Di);
+        vROL(Do, BCu, 1);
+        vxor(Do, BCi, Do);
+        vROL(Du, BCa, 1);
+        vxor(Du, BCo, Du);
+
+        vxor(Eba, Eba, Da);
+        vxor(Ege, Ege, De);
+        vROL(BCe, Ege, 44);
+        vxor(Eki, Eki, Di);
+        vROL(BCi, Eki, 43);
+        vxor(Emo, Emo, Do);
+        vROL(BCo, Emo, 21);
+        vxor(Esu, Esu, Du);
+        vROL(BCu, Esu, 14);
+        vXNA(Aba, Eba, BCe, BCi);
+        vxor(Aba, Aba, vdupq_n_u64(neon_KeccakF_RoundConstants[round + 1]));
+        vXNA(Abe, BCe, BCi, BCo);
+        vXNA(Abi, BCi, BCo, BCu);
+        vXNA(Abo, BCo, BCu, Eba);
+        vXNA(Abu, BCu, Eba, BCe);
+
+        vxor(Ebo, Ebo, Do);
+        vROL(BCa, Ebo, 28);
+        vxor(Egu, Egu, Du);
+        vROL(BCe, Egu, 20);
+        vxor(Eka, Eka, Da);
+        vROL(BCi, Eka, 3);
+        vxor(Eme, Eme, De);
+        vROL(BCo, Eme, 45);
+        vxor(Esi, Esi, Di);
+        vROL(BCu, Esi, 61);
+        vXNA(Aga, BCa, BCe, BCi);
+        vXNA(Age, BCe, BCi, BCo);
+        vXNA(Agi, BCi, BCo, BCu);
+        vXNA(Ago, BCo, BCu, BCa);
+        vXNA(Agu, BCu, BCa, BCe);
+
+        vxor(Ebe, Ebe, De);
+        vROL(BCa, Ebe, 1);
+        vxor(Egi, Egi, Di);
+        vROL(BCe, Egi, 6);
+        vxor(Eko, Eko, Do);
+        vROL(BCi, Eko, 25);
+        vxor(Emu, Emu, Du);
+        vROL(BCo, Emu, 8);
+        vxor(Esa, Esa, Da);
+        vROL(BCu, Esa, 18);
+        vXNA(Aka, BCa, BCe, BCi);
+        vXNA(Ake, BCe, BCi, BCo);
+        vXNA(Aki, BCi, BCo, BCu);
+        vXNA(Ako, BCo, BCu, BCa);
+        vXNA(Aku, BCu, BCa, BCe);
+
+        vxor(Ebu, Ebu, Du);
+        vROL(BCa, Ebu, 27);
+        vxor(Ega, Ega, Da);
+        vROL(BCe, Ega, 36);
+        vxor(Eke, Eke, De);
+        vROL(BCi, Eke, 10);
+        vxor(Emi, Emi, Di);
+        vROL(BCo, Emi, 15);
+        vxor(Eso, Eso, Do);
+        vROL(BCu, Eso, 56);
+        vXNA(Ama, BCa, BCe, BCi);
+        vXNA(Ame, BCe, BCi, BCo);
+        vXNA(Ami, BCi, BCo, BCu);
+        vXNA(Amo, BCo, BCu, BCa);
+        vXNA(Amu, BCu, BCa, BCe);
+
+        vxor(Ebi, Ebi, Di);
+        vROL(BCa, Ebi, 62);
+        vxor(Ego, Ego, Do);
+        vROL(BCe, Ego, 55);
+        vxor(Eku, Eku, Du);
+        vROL(BCi, Eku, 39);
+        vxor(Ema, Ema, Da);
+        vROL(BCo, Ema, 41);
+        vxor(Ese, Ese, De);
+        vROL(BCu, Ese, 2);
+        vXNA(Asa, BCa, BCe, BCi);
+        vXNA(Ase, BCe, BCi, BCo);
+        vXNA(Asi, BCi, BCo, BCu);
+        vXNA(Aso, BCo, BCu, BCa);
+        vXNA(Asu, BCu, BCa, BCe);
+    }
+
+    state[0] = Aba;
+    state[1] = Abe;
+    state[2] = Abi;
+    state[3] = Abo;
+    state[4] = Abu;
+    state[5] = Aga;
+    state[6] = Age;
+    state[7] = Agi;
+    state[8] = Ago;
+    state[9] = Agu;
+    state[10] = Aka;
+    state[11] = Ake;
+    state[12] = Aki;
+    state[13] = Ako;
+    state[14] = Aku;
+    state[15] = Ama;
+    state[16] = Ame;
+    state[17] = Ami;
+    state[18] = Amo;
+    state[19] = Amu;
+    state[20] = Asa;
+    state[21] = Ase;
+    state[22] = Asi;
+    state[23] = Aso;
+    state[24] = Asu;
+}
+
+/*************************************************
+* Name:        keccakx2_absorb
+*
+* Description: Absorb step of Keccak;
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - uint64_t *s: pointer to (uninitialized) output Keccak state
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - const uint8_t *m: pointer to input to be absorbed into s
+*              - size_t mlen: length of input in bytes
+*              - uint8_t p: domain-separation byte for different
+*                           Keccak-derived functions
+**************************************************/
+static
+void keccakx2_absorb(v128 s[25],
+                     unsigned int r,
+                     const uint8_t *in0,
+                     const uint8_t *in1,
+                     size_t inlen,
+                     uint8_t p) {
+    size_t i, pos = 0;
+
+    // Declare SIMD registers
+    v128 tmp, mask;
+    uint64x1_t a, b;
+    uint64x2_t a1, b1, atmp1, btmp1;
+    uint64x2x2_t a2, b2, atmp2, btmp2;
+    // End
+
+    for (i = 0; i < 25; ++i) {
+        s[i] = vdupq_n_u64(0);
+    }
+
+    // Load in0[i] to register, then in1[i] to register, exchange them
+    while (inlen >= r) {
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2 = vld1q_u64_x2((uint64_t *)&in0[pos]);
+            b2 = vld1q_u64_x2((uint64_t *)&in1[pos]);
+            // BD = zip1(AB and CD)
+            atmp2.val[0] = vzip1q_u64(a2.val[0], b2.val[0]);
+            atmp2.val[1] = vzip1q_u64(a2.val[1], b2.val[1]);
+            // AC = zip2(AB and CD)
+            btmp2.val[0] = vzip2q_u64(a2.val[0], b2.val[0]);
+            btmp2.val[1] = vzip2q_u64(a2.val[1], b2.val[1]);
+
+            vxor(s[i + 0], s[i + 0], atmp2.val[0]);
+            vxor(s[i + 1], s[i + 1], btmp2.val[0]);
+            vxor(s[i + 2], s[i + 2], atmp2.val[1]);
+            vxor(s[i + 3], s[i + 3], btmp2.val[1]);
+
+            pos += 8 * 2 * 2;
+        }
+        // Last iteration
+        i = r / 8 - 1;
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+        pos += 8;
+
+        KeccakF1600_StatePermutex2(s);
+        inlen -= r;
+    }
+
+    i = 0;
+    while (inlen >= 16) {
+        a1 = vld1q_u64((uint64_t *)&in0[pos]);
+        b1 = vld1q_u64((uint64_t *)&in1[pos]);
+        // BD = zip1(AB and CD)
+        atmp1 = vzip1q_u64(a1, b1);
+        // AC = zip2(AB and CD)
+        btmp1 = vzip2q_u64(a1, b1);
+
+        vxor(s[i + 0], s[i + 0], atmp1);
+        vxor(s[i + 1], s[i + 1], btmp1);
+
+        i += 2;
+        pos += 8 * 2;
+        inlen -= 8 * 2;
+    }
+
+    if (inlen >= 8) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        vxor(s[i], s[i], tmp);
+
+        i++;
+        pos += 8;
+        inlen -= 8;
+    }
+
+    if (inlen) {
+        a = vld1_u64((uint64_t *)&in0[pos]);
+        b = vld1_u64((uint64_t *)&in1[pos]);
+        tmp = vcombine_u64(a, b);
+        mask = vdupq_n_u64((1ULL << (8 * inlen)) - 1);
+        tmp = vandq_u64(tmp, mask);
+        vxor(s[i], s[i], tmp);
+    }
+
+    tmp = vdupq_n_u64((uint64_t)p << (8 * inlen));
+    vxor(s[i], s[i], tmp);
+
+    mask = vdupq_n_u64(1ULL << 63);
+    vxor(s[r / 8 - 1], s[r / 8 - 1], mask);
+}
+
+/*************************************************
+* Name:        keccak_squeezeblocks
+*
+* Description: Squeeze step of Keccak. Squeezes full blocks of r bytes each.
+*              Modifies the state. Can be called multiple times to keep
+*              squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out: pointer to output blocks
+*              - size_t nblocks: number of blocks to be squeezed (written to h)
+*              - unsigned int r: rate in bytes (e.g., 168 for SHAKE128)
+*              - uint64_t *s: pointer to input/output Keccak state
+**************************************************/
+static
+void keccakx2_squeezeblocks(uint8_t *out0,
+                            uint8_t *out1,
+                            size_t nblocks,
+                            unsigned int r,
+                            v128 s[25]) {
+    unsigned int i;
+
+    uint64x1_t a, b;
+    uint64x2x2_t a2, b2;
+
+    while (nblocks > 0) {
+        KeccakF1600_StatePermutex2(s);
+
+        for (i = 0; i < r / 8 - 1; i += 4) {
+            a2.val[0] = vuzp1q_u64(s[i], s[i + 1]);
+            b2.val[0] = vuzp2q_u64(s[i], s[i + 1]);
+            a2.val[1] = vuzp1q_u64(s[i + 2], s[i + 3]);
+            b2.val[1] = vuzp2q_u64(s[i + 2], s[i + 3]);
+            vst1q_u64_x2((uint64_t *)out0, a2);
+            vst1q_u64_x2((uint64_t *)out1, b2);
+
+            out0 += 32;
+            out1 += 32;
+        }
+
+        i = r / 8 - 1;
+        // Last iteration
+        a = vget_low_u64(s[i]);
+        b = vget_high_u64(s[i]);
+        vst1_u64((uint64_t *)out0, a);
+        vst1_u64((uint64_t *)out1, b);
+
+        out0 += 8;
+        out1 += 8;
+
+        --nblocks;
+    }
+}
+
+/*************************************************
+* Name:        shake128x2_absorb
+*
+* Description: Absorb step of the SHAKE128 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *state: pointer to (uninitialized) output
+*                                     Keccak state
+*              - const uint8_t *in:   pointer to input to be absorbed into s
+*              - size_t inlen:        length of input in bytes
+**************************************************/
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE128_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake128_squeezeblocks
+*
+* Description: Squeeze step of SHAKE128 XOF. Squeezes full blocks of
+*              SHAKE128_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE128_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake256_absorb
+*
+* Description: Absorb step of the SHAKE256 XOF.
+*              non-incremental, starts by zeroeing the state.
+*
+* Arguments:   - keccakx2_state *s:   pointer to (uninitialized) output Keccak state
+*              - const uint8_t *in: pointer to input to be absorbed into s
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen) {
+    keccakx2_absorb(state->s, SHAKE256_RATE, in0, in1, inlen, 0x1F);
+}
+
+/*************************************************
+* Name:        shake256_squeezeblocks
+*
+* Description: Squeeze step of SHAKE256 XOF. Squeezes full blocks of
+*              SHAKE256_RATE bytes each. Modifies the state. Can be called
+*              multiple times to keep squeezing, i.e., is incremental.
+*
+* Arguments:   - uint8_t *out:    pointer to output blocks
+*              - size_t nblocks:  number of blocks to be squeezed
+*                                 (written to output)
+*              - keccakx2_state *s: pointer to input/output Keccak state
+**************************************************/
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state) {
+    keccakx2_squeezeblocks(out0, out1, nblocks, SHAKE256_RATE, state->s);
+}
+
+/*************************************************
+* Name:        shake128
+*
+* Description: SHAKE128 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE128_RATE;
+    uint8_t t[2][SHAKE128_RATE];
+    keccakx2_state state;
+
+    shake128x2_absorb(&state, in0, in1, inlen);
+    shake128x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE128_RATE;
+    out1 += nblocks * SHAKE128_RATE;
+    outlen -= nblocks * SHAKE128_RATE;
+
+    if (outlen) {
+        shake128x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}
+
+/*************************************************
+* Name:        shake256
+*
+* Description: SHAKE256 XOF with non-incremental API
+*
+* Arguments:   - uint8_t *out:      pointer to output
+*              - size_t outlen:     requested output length in bytes
+*              - const uint8_t *in: pointer to input
+*              - size_t inlen:      length of input in bytes
+**************************************************/
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen) {
+    unsigned int i;
+    size_t nblocks = outlen / SHAKE256_RATE;
+    uint8_t t[2][SHAKE256_RATE];
+    keccakx2_state state;
+
+    shake256x2_absorb(&state, in0, in1, inlen);
+    shake256x2_squeezeblocks(out0, out1, nblocks, &state);
+
+    out0 += nblocks * SHAKE256_RATE;
+    out1 += nblocks * SHAKE256_RATE;
+    outlen -= nblocks * SHAKE256_RATE;
+
+    if (outlen) {
+        shake256x2_squeezeblocks(t[0], t[1], 1, &state);
+        for (i = 0; i < outlen; ++i) {
+            out0[i] = t[0][i];
+            out1[i] = t[1][i];
+        }
+    }
+}

src/sign/dilithium/dilithium5/aarch64/fips202x2.h

@@ -0,0 +1,60 @@
+#ifndef FIPS202X2_H
+#define FIPS202X2_H
+
+#include "params.h"
+#include <arm_neon.h>
+#include <stddef.h>
+
+typedef uint64x2_t v128;
+
+#define SHAKE128_RATE 168
+#define SHAKE256_RATE 136
+#define SHA3_256_RATE 136
+#define SHA3_512_RATE 72
+
+
+typedef struct {
+    v128 s[25];
+} keccakx2_state;
+
+
+#define shake128x2_absorb DILITHIUM_NAMESPACE(shake128x2_absorb)
+void shake128x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake128x2_squeezeblocks DILITHIUM_NAMESPACE(shake128x2_squeezeblocks)
+void shake128x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake256x2_absorb DILITHIUM_NAMESPACE(shake256x2_absorb)
+void shake256x2_absorb(keccakx2_state *state,
+                       const uint8_t *in0,
+                       const uint8_t *in1,
+                       size_t inlen);
+
+#define shake256x2_squeezeblocks DILITHIUM_NAMESPACE(shake256x2_squeezeblocks)
+void shake256x2_squeezeblocks(uint8_t *out0,
+                              uint8_t *out1,
+                              size_t nblocks,
+                              keccakx2_state *state);
+
+#define shake128x2 DILITHIUM_NAMESPACE(shake128x2)
+void shake128x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+
+#define shake256x2 DILITHIUM_NAMESPACE(shake256x2)
+void shake256x2(uint8_t *out0,
+                uint8_t *out1,
+                size_t outlen,
+                const uint8_t *in0,
+                const uint8_t *in1,
+                size_t inlen);
+#endif

src/sign/dilithium/dilithium5/aarch64/macros.inc

@@ -0,0 +1,84 @@
+#include "macros_common.inc"
+
+.macro wrap_trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3, qS, dD
+
+    trn1 \t0\qS, \a0\qS, \a1\qS
+    trn2 \t1\qS, \a0\qS, \a1\qS
+    trn1 \t2\qS, \a2\qS, \a3\qS
+    trn2 \t3\qS, \a2\qS, \a3\qS
+
+    trn1 \a0\dD, \t0\dD, \t2\dD
+    trn2 \a2\dD, \t0\dD, \t2\dD
+    trn1 \a1\dD, \t1\dD, \t3\dD
+    trn2 \a3\dD, \t1\dD, \t3\dD
+
+.endm
+
+.macro trn_4x4 a0, a1, a2, a3, t0, t1, t2, t3
+    wrap_trn_4x4 \a0, \a1, \a2, \a3, \t0, \t1, \t2, \t3, .4S, .2D
+.endm
+
+
+.macro dq_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1
+    wrap_dX_butterfly_vec_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \l0, \h0, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3
+    wrap_dX_butterfly_vec_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \l0, \h0, \l1, \h1, \l2, \h2, \l3, \h3, .4S, .S
+.endm
+
+
+.macro dq_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_top \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1
+    wrap_dX_butterfly_bot \a0, \a1, \b0, \b1, \t0, \t1, \mod, \z0, \l0, \h0, \z1, \l1, \h1, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro dq_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_dX_butterfly_mixed_rev \a0, \a1, \b0, \b1, \t0, \t1, \a2, \a3, \b2, \b3, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_montgomery_mul \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+
+.macro qq_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_top \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3
+    wrap_qX_butterfly_bot \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+.macro qq_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7
+    wrap_qX_butterfly_mixed_rev \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, \t0, \t1, \t2, \t3, \a4, \a5, \a6, \a7, \b4, \b5, \b6, \b7, \t4, \t5, \t6, \t7, \mod, \z0, \l0, \h0, \z1, \l1, \h1, \z2, \l2, \h2, \z3, \l3, \h3, \z4, \l4, \h4, \z5, \l5, \h5, \z6, \l6, \h6, \z7, \l7, \h7, .4S, .S
+.endm
+
+
+.macro qq_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q
+    wrap_qX_montgomery \c0, \c1, \c2, \c3, \l0, \l1, \l2, \l3, \h0, \h1, \h2, \h3, \t0, \t1, \t2, \t3, \Qprime, \Q, .2S, .4S, .2D
+.endm
+
+.macro qq_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3
+    wrap_qX_sub_add \s0, \s1, \s2, \s3, \t0, \t1, \t2, \t3, \a0, \a1, \a2, \a3, \b0, \b1, \b2, \b3, .4S
+.endm

src/sign/dilithium/dilithium5/aarch64/macros_common.inc

@@ -0,0 +1,423 @@
+// for ABI
+
+.macro push_all
+
+    sub sp, sp, #(16*9)
+    stp x19, x20, [sp, #16*0]
+    stp x21, x22, [sp, #16*1]
+    stp x23, x24, [sp, #16*2]
+    stp x25, x26, [sp, #16*3]
+    stp x27, x28, [sp, #16*4]
+    stp  d8,  d9, [sp, #16*5]
+    stp d10, d11, [sp, #16*6]
+    stp d12, d13, [sp, #16*7]
+    stp d14, d15, [sp, #16*8]
+
+.endm
+
+.macro pop_all
+
+    ldp x19, x20, [sp, #16*0]
+    ldp x21, x22, [sp, #16*1]
+    ldp x23, x24, [sp, #16*2]
+    ldp x25, x26, [sp, #16*3]
+    ldp x27, x28, [sp, #16*4]
+    ldp  d8,  d9, [sp, #16*5]
+    ldp d10, d11, [sp, #16*6]
+    ldp d12, d13, [sp, #16*7]
+    ldp d14, d15, [sp, #16*8]
+    add sp, sp, #(16*9)
+
+.endm
+
+// vector-scalar butterflies
+
+.macro wrap_dX_butterfly_top a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_bot a0, a1, b0, b1, t0, t1, mod, z0, l0, h0, z1, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_top a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_bot a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+    sub     \b2\wX, \a2\wX, \t2\wX
+    sub     \b3\wX, \a3\wX, \t3\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+    add     \a2\wX, \a2\wX, \t2\wX
+    add     \a3\wX, \a3\wX, \t3\wX
+
+.endm
+
+.macro wrap_qX_butterfly_mixed a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t4\wX, \b4\wX, \z4\nX[\h4]
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t5\wX, \b5\wX, \z5\nX[\h5]
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t6\wX, \b6\wX, \z6\nX[\h6]
+    sub      \b3\wX, \a3\wX, \t3\wX
+    mul      \t7\wX, \b7\wX, \z7\nX[\h7]
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b4\wX, \b4\wX, \z4\nX[\l4]
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b5\wX, \b5\wX, \z5\nX[\l5]
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b6\wX, \b6\wX, \z6\nX[\l6]
+    add      \a3\wX, \a3\wX, \t3\wX
+    sqrdmulh \b7\wX, \b7\wX, \z7\nX[\l7]
+
+    mls      \t4\wX, \b4\wX, \mod\nX[0]
+    mls      \t5\wX, \b5\wX, \mod\nX[0]
+    mls      \t6\wX, \b6\wX, \mod\nX[0]
+    mls      \t7\wX, \b7\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_qX_butterfly_mixed_rev a0, a1, a2, a3, b0, b1, b2, b3, t0, t1, t2, t3, a4, a5, a6, a7, b4, b5, b6, b7, t4, t5, t6, t7, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, z4, l4, h4, z5, l5, h5, z6, l6, h6, z7, l7, h7, wX, nX
+
+    mul      \t0\wX, \b0\wX, \z0\nX[\h0]
+    sub      \b4\wX, \a4\wX, \t4\wX
+    mul      \t1\wX, \b1\wX, \z1\nX[\h1]
+    sub      \b5\wX, \a5\wX, \t5\wX
+    mul      \t2\wX, \b2\wX, \z2\nX[\h2]
+    sub      \b6\wX, \a6\wX, \t6\wX
+    mul      \t3\wX, \b3\wX, \z3\nX[\h3]
+    sub      \b7\wX, \a7\wX, \t7\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \z0\nX[\l0]
+    add      \a4\wX, \a4\wX, \t4\wX
+    sqrdmulh \b1\wX, \b1\wX, \z1\nX[\l1]
+    add      \a5\wX, \a5\wX, \t5\wX
+    sqrdmulh \b2\wX, \b2\wX, \z2\nX[\l2]
+    add      \a6\wX, \a6\wX, \t6\wX
+    sqrdmulh \b3\wX, \b3\wX, \z3\nX[\l3]
+    add      \a7\wX, \a7\wX, \t7\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+// vector-vector butterflies
+
+.macro wrap_dX_butterfly_vec_top a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_bot a0, a1, b0, b1, t0, t1, mod, l0, h0, l1, h1, wX, nX
+
+    sub     \b0\wX, \a0\wX, \t0\wX
+    sub     \b1\wX, \a1\wX, \t1\wX
+
+    add     \a0\wX, \a0\wX, \t0\wX
+    add     \a1\wX, \a1\wX, \t1\wX
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    sub      \b0\wX, \a0\wX, \t0\wX
+    mul      \t2\wX, \b2\wX, \h2\wX
+    sub      \b1\wX, \a1\wX, \t1\wX
+    mul      \t3\wX, \b3\wX, \h3\wX
+
+    add      \a0\wX, \a0\wX, \t0\wX
+    sqrdmulh \b2\wX, \b2\wX, \l2\wX
+    add      \a1\wX, \a1\wX, \t1\wX
+    sqrdmulh \b3\wX, \b3\wX, \l3\wX
+
+    mls      \t2\wX, \b2\wX, \mod\nX[0]
+    mls      \t3\wX, \b3\wX, \mod\nX[0]
+
+.endm
+
+.macro wrap_dX_butterfly_vec_mixed_rev a0, a1, b0, b1, t0, t1, a2, a3, b2, b3, t2, t3, mod, l0, h0, l1, h1, l2, h2, l3, h3, wX, nX
+
+    mul      \t0\wX, \b0\wX, \h0\wX
+    sub      \b2\wX, \a2\wX, \t2\wX
+    mul      \t1\wX, \b1\wX, \h1\wX
+    sub      \b3\wX, \a3\wX, \t3\wX
+
+    sqrdmulh \b0\wX, \b0\wX, \l0\wX
+    add      \a2\wX, \a2\wX, \t2\wX
+    sqrdmulh \b1\wX, \b1\wX, \l1\wX
+    add      \a3\wX, \a3\wX, \t3\wX
+
+    mls      \t0\wX, \b0\wX, \mod\nX[0]
+    mls      \t1\wX, \b1\wX, \mod\nX[0]
+
+.endm
+
+// vector-scalar Barrett reduction
+
+.macro wrap_qX_barrett a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oX_barrett a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\nX[0]
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\nX[0]
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\nX[0]
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\nX[0]
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\nX[0]
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\nX[0]
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\nX[0]
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\nX[0]
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// vector-vector Barrett reduction
+
+.macro wrap_qo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+
+    srshr   \t2\wX, \t2\wX, \shrv
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+
+    mls     \a2\wX, \t2\wX, \Q\wX
+    mls     \a3\wX, \t3\wX, \Q\wX
+
+.endm
+
+.macro wrap_oo_barrett_vec a0, a1, a2, a3, t0, t1, t2, t3, a4, a5, a6, a7, t4, t5, t6, t7, barrett_const, shrv, Q, wX, nX
+
+    sqdmulh \t0\wX, \a0\wX, \barrett_const\wX
+    sqdmulh \t1\wX, \a1\wX, \barrett_const\wX
+    sqdmulh \t2\wX, \a2\wX, \barrett_const\wX
+    sqdmulh \t3\wX, \a3\wX, \barrett_const\wX
+
+    srshr   \t0\wX, \t0\wX, \shrv
+    sqdmulh \t4\wX, \a4\wX, \barrett_const\wX
+    srshr   \t1\wX, \t1\wX, \shrv
+    sqdmulh \t5\wX, \a5\wX, \barrett_const\wX
+    srshr   \t2\wX, \t2\wX, \shrv
+    sqdmulh \t6\wX, \a6\wX, \barrett_const\wX
+    srshr   \t3\wX, \t3\wX, \shrv
+    sqdmulh \t7\wX, \a7\wX, \barrett_const\wX
+
+    mls     \a0\wX, \t0\wX, \Q\wX
+    srshr   \t4\wX, \t4\wX, \shrv
+    mls     \a1\wX, \t1\wX, \Q\wX
+    srshr   \t5\wX, \t5\wX, \shrv
+    mls     \a2\wX, \t2\wX, \Q\wX
+    srshr   \t6\wX, \t6\wX, \shrv
+    mls     \a3\wX, \t3\wX, \Q\wX
+    srshr   \t7\wX, \t7\wX, \shrv
+
+    mls     \a4\wX, \t4\wX, \Q\wX
+    mls     \a5\wX, \t5\wX, \Q\wX
+    mls     \a6\wX, \t6\wX, \Q\wX
+    mls     \a7\wX, \t7\wX, \Q\wX
+
+.endm
+
+// Montgomery multiplication
+
+.macro wrap_qX_montgomery_mul b0, b1, b2, b3, t0, t1, t2, t3, mod, z0, l0, h0, z1, l1, h1, z2, l2, h2, z3, l3, h3, wX, nX
+
+    mul      \b0\wX, \t0\wX, \z0\nX[\h0]
+    mul      \b1\wX, \t1\wX, \z1\nX[\h1]
+    mul      \b2\wX, \t2\wX, \z2\nX[\h2]
+    mul      \b3\wX, \t3\wX, \z3\nX[\h3]
+
+    sqrdmulh \t0\wX, \t0\wX, \z0\nX[\l0]
+    sqrdmulh \t1\wX, \t1\wX, \z1\nX[\l1]
+    sqrdmulh \t2\wX, \t2\wX, \z2\nX[\l2]
+    sqrdmulh \t3\wX, \t3\wX, \z3\nX[\l3]
+
+    mls      \b0\wX, \t0\wX, \mod\nX[0]
+    mls      \b1\wX, \t1\wX, \mod\nX[0]
+    mls      \b2\wX, \t2\wX, \mod\nX[0]
+    mls      \b3\wX, \t3\wX, \mod\nX[0]
+
+.endm
+
+// Montgomery reduction with long
+
+.macro wrap_qX_montgomery c0, c1, c2, c3, l0, l1, l2, l3, h0, h1, h2, h3, t0, t1, t2, t3, Qprime, Q, lX, wX, dwX
+
+    uzp1 \t0\wX, \l0\wX, \h0\wX
+    uzp1 \t1\wX, \l1\wX, \h1\wX
+    uzp1 \t2\wX, \l2\wX, \h2\wX
+    uzp1 \t3\wX, \l3\wX, \h3\wX
+
+    mul \t0\wX, \t0\wX, \Qprime\wX
+    mul \t1\wX, \t1\wX, \Qprime\wX
+    mul \t2\wX, \t2\wX, \Qprime\wX
+    mul \t3\wX, \t3\wX, \Qprime\wX
+
+    smlal  \l0\dwX, \t0\lX, \Q\lX
+    smlal2 \h0\dwX, \t0\wX, \Q\wX
+    smlal  \l1\dwX, \t1\lX, \Q\lX
+    smlal2 \h1\dwX, \t1\wX, \Q\wX
+    smlal  \l2\dwX, \t2\lX, \Q\lX
+    smlal2 \h2\dwX, \t2\wX, \Q\wX
+    smlal  \l3\dwX, \t3\lX, \Q\lX
+    smlal2 \h3\dwX, \t3\wX, \Q\wX
+
+    uzp2 \c0\wX, \l0\wX, \h0\wX
+    uzp2 \c1\wX, \l1\wX, \h1\wX
+    uzp2 \c2\wX, \l2\wX, \h2\wX
+    uzp2 \c3\wX, \l3\wX, \h3\wX
+
+.endm
+
+// add_sub, sub_add
+
+.macro wrap_qX_add_sub s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s3\wX, \a3\wX, \b3\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+
+.endm
+
+.macro wrap_qX_sub_add s0, s1, s2, s3, t0, t1, t2, t3, a0, a1, a2, a3, b0, b1, b2, b3, wX
+
+    sub \t0\wX, \a0\wX, \b0\wX
+    add \s0\wX, \a0\wX, \b0\wX
+    sub \t1\wX, \a1\wX, \b1\wX
+    add \s1\wX, \a1\wX, \b1\wX
+    sub \t2\wX, \a2\wX, \b2\wX
+    add \s2\wX, \a2\wX, \b2\wX
+    sub \t3\wX, \a3\wX, \b3\wX
+    add \s3\wX, \a3\wX, \b3\wX
+
+.endm

src/sign/dilithium/dilithium5/aarch64/ntt.c

@@ -0,0 +1,35 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+#include <stdio.h>
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+
+/*************************************************
+* Name:        ntt
+*
+* Description: Forward NTT, in-place. No modular reduction is performed after
+*              additions or subtractions. Output vector is in bitreversed order.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void ntt(int32_t a[N]) {
+    NTT(a);
+}
+
+/*************************************************
+* Name:        invntt_tomont
+*
+* Description: Inverse NTT and multiplication by Montgomery factor 2^32.
+*              In-place. No modular reductions after additions or
+*              subtractions; input coefficients need to be smaller than
+*              Q in absolute value. Output coefficient are smaller than Q in
+*              absolute value.
+*
+* Arguments:   - uint32_t p[N]: input/output coefficient array
+**************************************************/
+void invntt_tomont(int32_t a[N]) {
+    iNTT(a);
+}

src/sign/dilithium/dilithium5/aarch64/packing.c

@@ -0,0 +1,261 @@
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+
+
+/*************************************************
+* Name:        pack_pk
+*
+* Description: Bit-pack public key pk = (rho, t1).
+*
+* Arguments:   - uint8_t pk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const polyveck *t1: pointer to vector t1
+**************************************************/
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const polyveck *t1) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        pk[i] = rho[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_pack(pk + i * POLYT1_PACKEDBYTES, &t1->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_pk
+*
+* Description: Unpack public key pk = (rho, t1).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const polyveck *t1: pointer to output vector t1
+*              - uint8_t pk[]: byte array containing bit-packed pk
+**************************************************/
+void unpack_pk(uint8_t rho[SEEDBYTES],
+               polyveck *t1,
+               const uint8_t pk[CRYPTO_PUBLICKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = pk[i];
+    }
+    pk += SEEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt1_unpack(&t1->vec[i], pk + i * POLYT1_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sk
+*
+* Description: Bit-pack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - uint8_t sk[]: output byte array
+*              - const uint8_t rho[]: byte array containing rho
+*              - const uint8_t tr[]: byte array containing tr
+*              - const uint8_t key[]: byte array containing key
+*              - const polyveck *t0: pointer to vector t0
+*              - const polyvecl *s1: pointer to vector s1
+*              - const polyveck *s2: pointer to vector s2
+**************************************************/
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = rho[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = key[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sk[i] = tr[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s1->vec[i]);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_pack(sk + i * POLYETA_PACKEDBYTES, &s2->vec[i]);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_pack(sk + i * POLYT0_PACKEDBYTES, &t0->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        unpack_sk
+*
+* Description: Unpack secret key sk = (rho, tr, key, t0, s1, s2).
+*
+* Arguments:   - const uint8_t rho[]: output byte array for rho
+*              - const uint8_t tr[]: output byte array for tr
+*              - const uint8_t key[]: output byte array for key
+*              - const polyveck *t0: pointer to output vector t0
+*              - const polyvecl *s1: pointer to output vector s1
+*              - const polyveck *s2: pointer to output vector s2
+*              - uint8_t sk[]: byte array containing bit-packed sk
+**************************************************/
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]) {
+    unsigned int i;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        rho[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        key[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        tr[i] = sk[i];
+    }
+    sk += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyeta_unpack(&s1->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += L * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyeta_unpack(&s2->vec[i], sk + i * POLYETA_PACKEDBYTES);
+    }
+    sk += K * POLYETA_PACKEDBYTES;
+
+    for (i = 0; i < K; ++i) {
+        polyt0_unpack(&t0->vec[i], sk + i * POLYT0_PACKEDBYTES);
+    }
+}
+
+/*************************************************
+* Name:        pack_sig
+*
+* Description: Bit-pack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t sig[]: output byte array
+*              - const uint8_t *c: pointer to challenge hash length SEEDBYTES
+*              - const polyvecl *z: pointer to vector z
+*              - const polyveck *h: pointer to hint vector h
+**************************************************/
+void pack_sig(uint8_t sig[CRYPTO_BYTES],
+              const uint8_t c[SEEDBYTES],
+              const polyvecl *z,
+              const polyveck *h) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        sig[i] = c[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_pack(sig + i * POLYZ_PACKEDBYTES, &z->vec[i]);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Encode h */
+    for (i = 0; i < OMEGA + K; ++i) {
+        sig[i] = 0;
+    }
+
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            if (h->vec[i].coeffs[j] != 0) {
+                sig[k++] = (uint8_t) j;
+            }
+        }
+
+        sig[OMEGA + i] = (uint8_t) k;
+    }
+}
+
+/*************************************************
+* Name:        unpack_sig
+*
+* Description: Unpack signature sig = (c, z, h).
+*
+* Arguments:   - uint8_t *c: pointer to output challenge hash
+*              - polyvecl *z: pointer to output vector z
+*              - polyveck *h: pointer to output hint vector h
+*              - const uint8_t sig[]: byte array containing
+*                bit-packed signature
+*
+* Returns 1 in case of malformed signature; otherwise 0.
+**************************************************/
+int unpack_sig(uint8_t c[SEEDBYTES],
+               polyvecl *z,
+               polyveck *h,
+               const uint8_t sig[CRYPTO_BYTES]) {
+    unsigned int i, j, k;
+
+    for (i = 0; i < SEEDBYTES; ++i) {
+        c[i] = sig[i];
+    }
+    sig += SEEDBYTES;
+
+    for (i = 0; i < L; ++i) {
+        polyz_unpack(&z->vec[i], sig + i * POLYZ_PACKEDBYTES);
+    }
+    sig += L * POLYZ_PACKEDBYTES;
+
+    /* Decode h */
+    k = 0;
+    for (i = 0; i < K; ++i) {
+        for (j = 0; j < N; ++j) {
+            h->vec[i].coeffs[j] = 0;
+        }
+
+        if (sig[OMEGA + i] < k || sig[OMEGA + i] > OMEGA) {
+            return 1;
+        }
+
+        for (j = k; j < sig[OMEGA + i]; ++j) {
+            /* Coefficients are ordered for strong unforgeability */
+            if (j > k && sig[j] <= sig[j - 1]) {
+                return 1;
+            }
+            h->vec[i].coeffs[sig[j]] = 1;
+        }
+
+        k = sig[OMEGA + i];
+    }
+
+    /* Extra indices are zero for strong unforgeability */
+    for (j = k; j < OMEGA; ++j) {
+        if (sig[j]) {
+            return 1;
+        }
+    }
+
+    return 0;
+}

src/sign/dilithium/dilithium5/aarch64/packing.h

@@ -0,0 +1,37 @@
+#ifndef PACKING_H
+#define PACKING_H
+#include "params.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#define pack_pk DILITHIUM_NAMESPACE(pack_pk)
+void pack_pk(uint8_t pk[CRYPTO_PUBLICKEYBYTES], const uint8_t rho[SEEDBYTES], const polyveck *t1);
+
+#define pack_sk DILITHIUM_NAMESPACE(pack_sk)
+void pack_sk(uint8_t sk[CRYPTO_SECRETKEYBYTES],
+             const uint8_t rho[SEEDBYTES],
+             const uint8_t tr[SEEDBYTES],
+             const uint8_t key[SEEDBYTES],
+             const polyveck *t0,
+             const polyvecl *s1,
+             const polyveck *s2);
+
+#define pack_sig DILITHIUM_NAMESPACE(pack_sig)
+void pack_sig(uint8_t sig[CRYPTO_BYTES], const uint8_t c[SEEDBYTES], const polyvecl *z, const polyveck *h);
+
+#define unpack_pk DILITHIUM_NAMESPACE(unpack_pk)
+void unpack_pk(uint8_t rho[SEEDBYTES], polyveck *t1, const uint8_t pk[CRYPTO_PUBLICKEYBYTES]);
+
+#define unpack_sk DILITHIUM_NAMESPACE(unpack_sk)
+void unpack_sk(uint8_t rho[SEEDBYTES],
+               uint8_t tr[SEEDBYTES],
+               uint8_t key[SEEDBYTES],
+               polyveck *t0,
+               polyvecl *s1,
+               polyveck *s2,
+               const uint8_t sk[CRYPTO_SECRETKEYBYTES]);
+
+#define unpack_sig DILITHIUM_NAMESPACE(unpack_sig)
+int unpack_sig(uint8_t c[SEEDBYTES], polyvecl *z, polyveck *h, const uint8_t sig[CRYPTO_BYTES]);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/params.h

@@ -0,0 +1,51 @@
+#ifndef PARAMS_H
+#define PARAMS_H
+
+//#define DILITHIUM_MODE 2
+//#define DILITHIUM_MODE 3
+#define DILITHIUM_MODE 5
+
+#define CRYPTO_NAMESPACETOP PQCLEAN_DILITHIUM5_AARCH64_crypto_sign
+#define CRYPTO_NAMESPACE(s) PQCLEAN_DILITHIUM5_AARCH64_##s
+#define DILITHIUM_NAMESPACETOP CRYPTO_NAMESPACETOP
+#define DILITHIUM_NAMESPACE(s) CRYPTO_NAMESPACE(s)
+
+
+#define SEEDBYTES 32
+#define CRHBYTES 64
+#define N 256
+#define DILITHIUM_Q 8380417
+#define D 13
+#define ROOT_OF_UNITY 1753
+
+
+#define K 8
+#define L 7
+#define ETA 2
+#define TAU 60
+#define BETA 120
+#define GAMMA1 (1 << 19)
+#define GAMMA2 ((DILITHIUM_Q-1)/32)
+#define OMEGA 75
+#define CRYPTO_ALGNAME "Dilithium5"
+
+
+#define POLYT1_PACKEDBYTES  320
+#define POLYT0_PACKEDBYTES  416
+#define POLYVECH_PACKEDBYTES (OMEGA + K)
+
+
+#define POLYZ_PACKEDBYTES   640
+
+#define POLYW1_PACKEDBYTES  128
+
+#define POLYETA_PACKEDBYTES  96
+
+#define CRYPTO_PUBLICKEYBYTES (SEEDBYTES + K*POLYT1_PACKEDBYTES)
+#define CRYPTO_SECRETKEYBYTES (3*SEEDBYTES \
+                               + L*POLYETA_PACKEDBYTES \
+                               + K*POLYETA_PACKEDBYTES \
+                               + K*POLYT0_PACKEDBYTES)
+#define CRYPTO_BYTES (SEEDBYTES + L*POLYZ_PACKEDBYTES + POLYVECH_PACKEDBYTES)
+
+#endif

src/sign/dilithium/dilithium5/aarch64/poly.c

@@ -0,0 +1,915 @@
+#include "params.h"
+#include "poly.h"
+#include "reduce.h"
+#include "rounding.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+#include "fips202x2.h"
+
+#include "NTT_params.h"
+#include "ntt.h"
+
+static const int32_t montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+#define DBENCH_START()
+#define DBENCH_STOP(t)
+
+/*************************************************
+* Name:        poly_reduce
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              representative in [-6283009,6283007].
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_poly_reduce(int32_t *, const int32_t *);
+void poly_reduce(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_poly_reduce(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_caddq
+*
+* Description: For all coefficients of in/out polynomial add Q if
+*              coefficient is negative.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_poly_caddq(int32_t *, const int32_t *);
+void poly_caddq(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_poly_caddq(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_freeze
+*
+* Description: Inplace reduction of all coefficients of polynomial to
+*              standard representatives.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_poly_freeze(int32_t *, const int32_t *);
+void poly_freeze(poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_poly_freeze(a->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tred);
+}
+
+/*************************************************
+* Name:        poly_add
+*
+* Description: Add polynomials. No modular reduction is performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first summand
+*              - const poly *b: pointer to second summand
+**************************************************/
+void poly_add(poly *c, const poly *a, const poly *b)  {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] + b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_sub
+*
+* Description: Subtract polynomials. No modular reduction is
+*              performed.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial to be
+*                               subtraced from first input polynomial
+**************************************************/
+void poly_sub(poly *c, const poly *a, const poly *b) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = a->coeffs[i] - b->coeffs[i];
+    }
+
+    DBENCH_STOP(*tadd);
+}
+
+/*************************************************
+* Name:        poly_shiftl
+*
+* Description: Multiply polynomial by 2^D without modular reduction. Assumes
+*              input coefficients to be less than 2^{31-D} in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_shiftl(poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a->coeffs[i] <<= D;
+    }
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_ntt
+*
+* Description: Inplace forward NTT. Coefficients can grow by
+*              8*Q in absolute value.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_ntt(poly *a) {
+    DBENCH_START();
+
+    ntt(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_invntt_tomont
+*
+* Description: Inplace inverse NTT and multiplication by 2^{32}.
+*              Input coefficients need to be less than Q in absolute
+*              value and output coefficients are again bounded by Q.
+*
+* Arguments:   - poly *a: pointer to input/output polynomial
+**************************************************/
+void poly_invntt_tomont(poly *a) {
+    DBENCH_START();
+
+    invntt_tomont(a->coeffs);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_pointwise_montgomery
+*
+* Description: Pointwise multiplication of polynomials in NTT domain
+*              representation and multiplication of resulting polynomial
+*              by 2^{-32}.
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const poly *a: pointer to first input polynomial
+*              - const poly *b: pointer to second input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_poly_pointwise_montgomery(int32_t *des, const int32_t *src1, const int32_t *src2, const int32_t *table);
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_poly_pointwise_montgomery(c->coeffs, a->coeffs, b->coeffs, montgomery_const);
+
+    DBENCH_STOP(*tmul);
+}
+
+/*************************************************
+* Name:        poly_power2round
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute c0, c1 such that c mod Q = c1*2^D + c0
+*              with -2^{D-1} < c0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_poly_power2round(int32_t *, int32_t *, const int32_t *);
+void poly_power2round(poly *a1, poly *a0, const poly *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_poly_power2round(a1->coeffs, a0->coeffs, a->coeffs);
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_decompose
+*
+* Description: For all coefficients c of the input polynomial,
+*              compute high and low bits c0, c1 such c mod Q = c1*ALPHA + c0
+*              with -ALPHA/2 < c0 <= ALPHA/2 except c1 = (Q-1)/ALPHA where we
+*              set c1 = 0 and -ALPHA/2 <= c0 = c mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - poly *a1: pointer to output polynomial with coefficients c1
+*              - poly *a0: pointer to output polynomial with coefficients c0
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void poly_decompose(poly *a1, poly *a0, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        a1->coeffs[i] = decompose(&a0->coeffs[i], a->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_make_hint
+*
+* Description: Compute hint polynomial. The coefficients of which indicate
+*              whether the low bits of the corresponding coefficient of
+*              the input polynomial overflow into the high bits.
+*
+* Arguments:   - poly *h: pointer to output hint polynomial
+*              - const poly *a0: pointer to low part of input polynomial
+*              - const poly *a1: pointer to high part of input polynomial
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1) {
+    unsigned int i, s = 0;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        h->coeffs[i] = make_hint(a0->coeffs[i], a1->coeffs[i]);
+        s += h->coeffs[i];
+    }
+
+    DBENCH_STOP(*tround);
+    return s;
+}
+
+/*************************************************
+* Name:        poly_use_hint
+*
+* Description: Use hint polynomial to correct the high bits of a polynomial.
+*
+* Arguments:   - poly *b: pointer to output polynomial with corrected high bits
+*              - const poly *a: pointer to input polynomial
+*              - const poly *h: pointer to input hint polynomial
+**************************************************/
+void poly_use_hint(poly *b, const poly *a, const poly *h) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N; ++i) {
+        b->coeffs[i] = use_hint(a->coeffs[i], h->coeffs[i]);
+    }
+
+    DBENCH_STOP(*tround);
+}
+
+/*************************************************
+* Name:        poly_chknorm
+*
+* Description: Check infinity norm of polynomial against given bound.
+*              Assumes input coefficients were reduced by reduce32().
+*
+* Arguments:   - const poly *a: pointer to polynomial
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm is strictly smaller than B <= (Q-1)/8 and 1 otherwise.
+**************************************************/
+int poly_chknorm(const poly *a, int32_t B) {
+    unsigned int i;
+    int32_t t;
+    DBENCH_START();
+
+    if (B > (DILITHIUM_Q - 1) / 8) {
+        return 1;
+    }
+
+    /* It is ok to leak which coefficient violates the bound since
+       the probability for each coefficient is independent of secret
+       data but we must not leak the sign of the centralized representative. */
+    for (i = 0; i < N; ++i) {
+        /* Absolute value */
+        t = a->coeffs[i] >> 31;
+        t = a->coeffs[i] - (t & 2 * a->coeffs[i]);
+
+        if (t >= B) {
+            DBENCH_STOP(*tsample);
+            return 1;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return 0;
+}
+
+/*************************************************
+* Name:        rej_uniform
+*
+* Description: Sample uniformly random coefficients in [0, Q-1] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_uniform(int32_t *a,
+                                unsigned int len,
+                                const uint8_t *buf,
+                                unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos + 3 <= buflen) {
+        t  = buf[pos++];
+        t |= (uint32_t)buf[pos++] << 8;
+        t |= (uint32_t)buf[pos++] << 16;
+        t &= 0x7FFFFF;
+
+        if (t < DILITHIUM_Q) {
+            a[ctr++] = t;
+        }
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [0,Q-1] by performing rejection sampling on the
+*              output stream of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length SEEDBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+
+#define POLY_UNIFORM_NBLOCKS ((768 + STREAM128_BLOCKBYTES - 1)/STREAM128_BLOCKBYTES)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce) {
+    unsigned int i, ctr, off;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    stream128_state state;
+
+    stream128_init(&state, seed, nonce);
+    stream128_squeezeblocks(buf, POLY_UNIFORM_NBLOCKS, &state);
+
+    ctr = rej_uniform(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        off = buflen % 3;
+        for (i = 0; i < off; ++i) {
+            buf[i] = buf[buflen - off + i];
+        }
+
+        stream128_squeezeblocks(buf + off, 1, &state);
+        buflen = STREAM128_BLOCKBYTES + off;
+        ctr += rej_uniform(a->coeffs + ctr, N - ctr, buf, buflen);
+    }
+    stream128_release(&state);
+}
+
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES;
+    uint8_t buf0[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+    uint8_t buf1[POLY_UNIFORM_NBLOCKS * STREAM128_BLOCKBYTES + 2];
+
+    keccakx2_state statex2;
+    dilithium_shake128x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake128x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_NBLOCKS, &statex2);
+
+    ctr0 = rej_uniform(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_uniform(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake128x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_uniform(a0->coeffs + ctr0, N - ctr0, buf0, buflen);
+        ctr1 += rej_uniform(a1->coeffs + ctr1, N - ctr1, buf1, buflen);
+    }
+
+
+}
+
+/*************************************************
+* Name:        rej_eta
+*
+* Description: Sample uniformly random coefficients in [-ETA, ETA] by
+*              performing rejection sampling on array of random bytes.
+*
+* Arguments:   - int32_t *a: pointer to output array (allocated)
+*              - unsigned int len: number of coefficients to be sampled
+*              - const uint8_t *buf: array of random bytes
+*              - unsigned int buflen: length of array of random bytes
+*
+* Returns number of sampled coefficients. Can be smaller than len if not enough
+* random bytes were given.
+**************************************************/
+static unsigned int rej_eta(int32_t *a,
+                            unsigned int len,
+                            const uint8_t *buf,
+                            unsigned int buflen) {
+    unsigned int ctr, pos;
+    uint32_t t0, t1;
+    DBENCH_START();
+
+    ctr = pos = 0;
+    while (ctr < len && pos < buflen) {
+        t0 = buf[pos] & 0x0F;
+        t1 = buf[pos++] >> 4;
+
+
+        if (t0 < 15) {
+            t0 = t0 - (205 * t0 >> 10) * 5;
+            a[ctr++] = 2 - t0;
+        }
+        if (t1 < 15 && ctr < len) {
+            t1 = t1 - (205 * t1 >> 10) * 5;
+            a[ctr++] = 2 - t1;
+        }
+
+
+
+    }
+
+    DBENCH_STOP(*tsample);
+    return ctr;
+}
+
+/*************************************************
+* Name:        poly_uniform_eta
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-ETA,ETA] by performing rejection sampling on the
+*              output stream from SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 2-byte nonce
+**************************************************/
+#define POLY_UNIFORM_ETA_NBLOCKS ((136 + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce) {
+    unsigned int ctr;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+    uint8_t buf[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_ETA_NBLOCKS, &state);
+
+    ctr = rej_eta(a->coeffs, N, buf, buflen);
+
+    while (ctr < N) {
+        stream256_squeezeblocks(buf, 1, &state);
+        ctr += rej_eta(a->coeffs + ctr, N - ctr, buf, STREAM256_BLOCKBYTES);
+    }
+    stream256_release(&state);
+}
+
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1) {
+    unsigned int ctr0, ctr1;
+    unsigned int buflen = POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES;
+
+    uint8_t buf0[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_ETA_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_ETA_NBLOCKS, &statex2);
+
+    ctr0 = rej_eta(a0->coeffs, N, buf0, buflen);
+    ctr1 = rej_eta(a1->coeffs, N, buf1, buflen);
+
+    while (ctr0 < N || ctr1 < N) {
+        shake256x2_squeezeblocks(buf0, buf1, 1, &statex2);
+        ctr0 += rej_eta(a0->coeffs + ctr0, N - ctr0, buf0, STREAM256_BLOCKBYTES);
+        ctr1 += rej_eta(a1->coeffs + ctr1, N - ctr1, buf1, STREAM256_BLOCKBYTES);
+    }
+}
+
+/*************************************************
+* Name:        poly_uniform_gamma1m1
+*
+* Description: Sample polynomial with uniformly random coefficients
+*              in [-(GAMMA1 - 1), GAMMA1] by unpacking output stream
+*              of SHAKE256(seed|nonce) or AES256CTR(seed,nonce).
+*
+* Arguments:   - poly *a: pointer to output polynomial
+*              - const uint8_t seed[]: byte array with seed of length CRHBYTES
+*              - uint16_t nonce: 16-bit nonce
+**************************************************/
+#define POLY_UNIFORM_GAMMA1_NBLOCKS ((POLYZ_PACKEDBYTES + STREAM256_BLOCKBYTES - 1)/STREAM256_BLOCKBYTES)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce) {
+    uint8_t buf[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    stream256_state state;
+
+    stream256_init(&state, seed, nonce);
+    stream256_squeezeblocks(buf, POLY_UNIFORM_GAMMA1_NBLOCKS, &state);
+    stream256_release(&state);
+    polyz_unpack(a, buf);
+}
+
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1) {
+
+    uint8_t buf0[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+    uint8_t buf1[POLY_UNIFORM_GAMMA1_NBLOCKS * STREAM256_BLOCKBYTES];
+
+    keccakx2_state statex2;
+
+    dilithium_shake256x2_stream_init(&statex2, seed, nonce0, nonce1);
+    shake256x2_squeezeblocks(buf0, buf1, POLY_UNIFORM_GAMMA1_NBLOCKS, &statex2);
+
+    polyz_unpack(a0, buf0);
+    polyz_unpack(a1, buf1);
+
+}
+
+/*************************************************
+* Name:        challenge
+*
+* Description: Implementation of H. Samples polynomial with TAU nonzero
+*              coefficients in {-1,1} using the output stream of
+*              SHAKE256(seed).
+*
+* Arguments:   - poly *c: pointer to output polynomial
+*              - const uint8_t mu[]: byte array containing seed of length SEEDBYTES
+**************************************************/
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]) {
+    unsigned int i, b, pos;
+    uint64_t signs;
+    uint8_t buf[SHAKE256_RATE];
+    shake256incctx state;
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, seed, SEEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(buf, sizeof buf, &state);
+
+    signs = 0;
+    for (i = 0; i < 8; ++i) {
+        signs |= (uint64_t)buf[i] << 8 * i;
+    }
+    pos = 8;
+
+    for (i = 0; i < N; ++i) {
+        c->coeffs[i] = 0;
+    }
+    for (i = N - TAU; i < N; ++i) {
+        do {
+            if (pos >= SHAKE256_RATE) {
+                shake256_inc_squeeze(buf, sizeof buf, &state);
+                pos = 0;
+            }
+
+            b = buf[pos++];
+        } while (b > i);
+
+        c->coeffs[i] = c->coeffs[b];
+        c->coeffs[b] = 1 - 2 * (signs & 1);
+        signs >>= 1;
+    }
+    shake256_inc_ctx_release(&state);
+}
+
+/*************************************************
+* Name:        polyeta_pack
+*
+* Description: Bit-pack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYETA_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyeta_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint8_t t[8];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 8; ++i) {
+        t[0] = ETA - a->coeffs[8 * i + 0];
+        t[1] = ETA - a->coeffs[8 * i + 1];
+        t[2] = ETA - a->coeffs[8 * i + 2];
+        t[3] = ETA - a->coeffs[8 * i + 3];
+        t[4] = ETA - a->coeffs[8 * i + 4];
+        t[5] = ETA - a->coeffs[8 * i + 5];
+        t[6] = ETA - a->coeffs[8 * i + 6];
+        t[7] = ETA - a->coeffs[8 * i + 7];
+
+        r[3 * i + 0]  = (t[0] >> 0) | (t[1] << 3) | (t[2] << 6);
+        r[3 * i + 1]  = (t[2] >> 2) | (t[3] << 1) | (t[4] << 4) | (t[5] << 7);
+        r[3 * i + 2]  = (t[5] >> 1) | (t[6] << 2) | (t[7] << 5);
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyeta_unpack
+*
+* Description: Unpack polynomial with coefficients in [-ETA,ETA].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyeta_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 8; ++i) {
+        r->coeffs[8 * i + 0] =  (a[3 * i + 0] >> 0) & 7;
+        r->coeffs[8 * i + 1] =  (a[3 * i + 0] >> 3) & 7;
+        r->coeffs[8 * i + 2] = ((a[3 * i + 0] >> 6) | (a[3 * i + 1] << 2)) & 7;
+        r->coeffs[8 * i + 3] =  (a[3 * i + 1] >> 1) & 7;
+        r->coeffs[8 * i + 4] =  (a[3 * i + 1] >> 4) & 7;
+        r->coeffs[8 * i + 5] = ((a[3 * i + 1] >> 7) | (a[3 * i + 2] << 1)) & 7;
+        r->coeffs[8 * i + 6] =  (a[3 * i + 2] >> 2) & 7;
+        r->coeffs[8 * i + 7] =  (a[3 * i + 2] >> 5) & 7;
+
+        r->coeffs[8 * i + 0] = ETA - r->coeffs[8 * i + 0];
+        r->coeffs[8 * i + 1] = ETA - r->coeffs[8 * i + 1];
+        r->coeffs[8 * i + 2] = ETA - r->coeffs[8 * i + 2];
+        r->coeffs[8 * i + 3] = ETA - r->coeffs[8 * i + 3];
+        r->coeffs[8 * i + 4] = ETA - r->coeffs[8 * i + 4];
+        r->coeffs[8 * i + 5] = ETA - r->coeffs[8 * i + 5];
+        r->coeffs[8 * i + 6] = ETA - r->coeffs[8 * i + 6];
+        r->coeffs[8 * i + 7] = ETA - r->coeffs[8 * i + 7];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_pack
+*
+* Description: Bit-pack polynomial t1 with coefficients fitting in 10 bits.
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 4; ++i) {
+        r[5 * i + 0] = (uint8_t) (a->coeffs[4 * i + 0] >> 0);
+        r[5 * i + 1] = (uint8_t) ((a->coeffs[4 * i + 0] >> 8) | (a->coeffs[4 * i + 1] << 2));
+        r[5 * i + 2] = (uint8_t) ((a->coeffs[4 * i + 1] >> 6) | (a->coeffs[4 * i + 2] << 4));
+        r[5 * i + 3] = (uint8_t) ((a->coeffs[4 * i + 2] >> 4) | (a->coeffs[4 * i + 3] << 6));
+        r[5 * i + 4] = (uint8_t) (a->coeffs[4 * i + 3] >> 2);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt1_unpack
+*
+* Description: Unpack polynomial t1 with 10-bit coefficients.
+*              Output coefficients are standard representatives.
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_10_to_32(int32_t *, const uint8_t *);
+void polyt1_unpack(poly *r, const uint8_t *a) {
+    DBENCH_START();
+
+    PQCLEAN_DILITHIUM5_AARCH64_asm_10_to_32(r->coeffs, a);
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_pack
+*
+* Description: Bit-pack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYT0_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyt0_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[8];
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        t[0] = (1 << (D - 1)) - a->coeffs[8 * i + 0];
+        t[1] = (1 << (D - 1)) - a->coeffs[8 * i + 1];
+        t[2] = (1 << (D - 1)) - a->coeffs[8 * i + 2];
+        t[3] = (1 << (D - 1)) - a->coeffs[8 * i + 3];
+        t[4] = (1 << (D - 1)) - a->coeffs[8 * i + 4];
+        t[5] = (1 << (D - 1)) - a->coeffs[8 * i + 5];
+        t[6] = (1 << (D - 1)) - a->coeffs[8 * i + 6];
+        t[7] = (1 << (D - 1)) - a->coeffs[8 * i + 7];
+
+        r[13 * i + 0]  =  (uint8_t) t[0];
+        r[13 * i + 1]  =  (uint8_t) (t[0] >>  8);
+        r[13 * i + 1] |=  (uint8_t) (t[1] <<  5);
+        r[13 * i + 2]  =  (uint8_t) (t[1] >>  3);
+        r[13 * i + 3]  =  (uint8_t) (t[1] >> 11);
+        r[13 * i + 3] |=  (uint8_t) (t[2] <<  2);
+        r[13 * i + 4]  =  (uint8_t) (t[2] >>  6);
+        r[13 * i + 4] |=  (uint8_t) (t[3] <<  7);
+        r[13 * i + 5]  =  (uint8_t) (t[3] >>  1);
+        r[13 * i + 6]  =  (uint8_t) (t[3] >>  9);
+        r[13 * i + 6] |=  (uint8_t) (t[4] <<  4);
+        r[13 * i + 7]  =  (uint8_t) (t[4] >>  4);
+        r[13 * i + 8]  =  (uint8_t) (t[4] >> 12);
+        r[13 * i + 8] |=  (uint8_t) (t[5] <<  1);
+        r[13 * i + 9]  =  (uint8_t) (t[5] >>  7);
+        r[13 * i + 9] |=  (uint8_t) (t[6] <<  6);
+        r[13 * i + 10]  =  (uint8_t) (t[6] >>  2);
+        r[13 * i + 11]  =  (uint8_t) (t[6] >> 10);
+        r[13 * i + 11] |=  (uint8_t) (t[7] <<  3);
+        r[13 * i + 12]  =  (uint8_t) (t[7] >>  5);
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyt0_unpack
+*
+* Description: Unpack polynomial t0 with coefficients in ]-2^{D-1}, 2^{D-1}].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyt0_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+    for (i = 0; i < N / 8; ++i) {
+        r->coeffs[8 * i + 0]  = a[13 * i + 0];
+        r->coeffs[8 * i + 0] |= (uint32_t)a[13 * i + 1] << 8;
+        r->coeffs[8 * i + 0] &= 0x1FFF;
+
+        r->coeffs[8 * i + 1]  = a[13 * i + 1] >> 5;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 2] << 3;
+        r->coeffs[8 * i + 1] |= (uint32_t)a[13 * i + 3] << 11;
+        r->coeffs[8 * i + 1] &= 0x1FFF;
+
+        r->coeffs[8 * i + 2]  = a[13 * i + 3] >> 2;
+        r->coeffs[8 * i + 2] |= (uint32_t)a[13 * i + 4] << 6;
+        r->coeffs[8 * i + 2] &= 0x1FFF;
+
+        r->coeffs[8 * i + 3]  = a[13 * i + 4] >> 7;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 5] << 1;
+        r->coeffs[8 * i + 3] |= (uint32_t)a[13 * i + 6] << 9;
+        r->coeffs[8 * i + 3] &= 0x1FFF;
+
+        r->coeffs[8 * i + 4]  = a[13 * i + 6] >> 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 7] << 4;
+        r->coeffs[8 * i + 4] |= (uint32_t)a[13 * i + 8] << 12;
+        r->coeffs[8 * i + 4] &= 0x1FFF;
+
+        r->coeffs[8 * i + 5]  = a[13 * i + 8] >> 1;
+        r->coeffs[8 * i + 5] |= (uint32_t)a[13 * i + 9] << 7;
+        r->coeffs[8 * i + 5] &= 0x1FFF;
+
+        r->coeffs[8 * i + 6]  = a[13 * i + 9] >> 6;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 10] << 2;
+        r->coeffs[8 * i + 6] |= (uint32_t)a[13 * i + 11] << 10;
+        r->coeffs[8 * i + 6] &= 0x1FFF;
+
+        r->coeffs[8 * i + 7]  = a[13 * i + 11] >> 3;
+        r->coeffs[8 * i + 7] |= (uint32_t)a[13 * i + 12] << 5;
+        r->coeffs[8 * i + 7] &= 0x1FFF;
+
+        r->coeffs[8 * i + 0] = (1 << (D - 1)) - r->coeffs[8 * i + 0];
+        r->coeffs[8 * i + 1] = (1 << (D - 1)) - r->coeffs[8 * i + 1];
+        r->coeffs[8 * i + 2] = (1 << (D - 1)) - r->coeffs[8 * i + 2];
+        r->coeffs[8 * i + 3] = (1 << (D - 1)) - r->coeffs[8 * i + 3];
+        r->coeffs[8 * i + 4] = (1 << (D - 1)) - r->coeffs[8 * i + 4];
+        r->coeffs[8 * i + 5] = (1 << (D - 1)) - r->coeffs[8 * i + 5];
+        r->coeffs[8 * i + 6] = (1 << (D - 1)) - r->coeffs[8 * i + 6];
+        r->coeffs[8 * i + 7] = (1 << (D - 1)) - r->coeffs[8 * i + 7];
+    }
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_pack
+*
+* Description: Bit-pack polynomial with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYZ_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyz_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    uint32_t t[4];
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        t[0] = GAMMA1 - a->coeffs[2 * i + 0];
+        t[1] = GAMMA1 - a->coeffs[2 * i + 1];
+
+        r[5 * i + 0]  = t[0];
+        r[5 * i + 1]  = t[0] >> 8;
+        r[5 * i + 2]  = t[0] >> 16;
+        r[5 * i + 2] |= t[1] << 4;
+        r[5 * i + 3]  = t[1] >> 4;
+        r[5 * i + 4]  = t[1] >> 12;
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyz_unpack
+*
+* Description: Unpack polynomial z with coefficients
+*              in [-(GAMMA1 - 1), GAMMA1].
+*
+* Arguments:   - poly *r: pointer to output polynomial
+*              - const uint8_t *a: byte array with bit-packed polynomial
+**************************************************/
+void polyz_unpack(poly *r, const uint8_t *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        r->coeffs[2 * i + 0]  = a[5 * i + 0];
+        r->coeffs[2 * i + 0] |= (uint32_t)a[5 * i + 1] << 8;
+        r->coeffs[2 * i + 0] |= (uint32_t)a[5 * i + 2] << 16;
+        r->coeffs[2 * i + 0] &= 0xFFFFF;
+
+        r->coeffs[2 * i + 1]  = a[5 * i + 2] >> 4;
+        r->coeffs[2 * i + 1] |= (uint32_t)a[5 * i + 3] << 4;
+        r->coeffs[2 * i + 1] |= (uint32_t)a[5 * i + 4] << 12;
+        r->coeffs[2 * i + 0] &= 0xFFFFF;
+
+        r->coeffs[2 * i + 0] = GAMMA1 - r->coeffs[2 * i + 0];
+        r->coeffs[2 * i + 1] = GAMMA1 - r->coeffs[2 * i + 1];
+    }
+
+
+    DBENCH_STOP(*tpack);
+}
+
+/*************************************************
+* Name:        polyw1_pack
+*
+* Description: Bit-pack polynomial w1 with coefficients in [0,15] or [0,43].
+*              Input coefficients are assumed to be standard representatives.
+*
+* Arguments:   - uint8_t *r: pointer to output byte array with at least
+*                            POLYW1_PACKEDBYTES bytes
+*              - const poly *a: pointer to input polynomial
+**************************************************/
+void polyw1_pack(uint8_t *r, const poly *a) {
+    unsigned int i;
+    DBENCH_START();
+
+
+    for (i = 0; i < N / 2; ++i) {
+        r[i] = a->coeffs[2 * i + 0] | (a->coeffs[2 * i + 1] << 4);
+    }
+
+
+    DBENCH_STOP(*tpack);
+}

src/sign/dilithium/dilithium5/aarch64/poly.h

@@ -0,0 +1,92 @@
+#ifndef POLY_H
+#define POLY_H
+#include "params.h"
+#include <stdint.h>
+
+typedef struct {
+    int32_t coeffs[N];
+} poly;
+
+#define poly_reduce DILITHIUM_NAMESPACE(poly_reduce)
+void poly_reduce(poly *a);
+#define poly_caddq DILITHIUM_NAMESPACE(poly_caddq)
+void poly_caddq(poly *a);
+#define poly_freeze DILITHIUM_NAMESPACE(poly_freeze)
+void poly_freeze(poly *a);
+
+#define poly_add DILITHIUM_NAMESPACE(poly_add)
+void poly_add(poly *c, const poly *a, const poly *b);
+#define poly_sub DILITHIUM_NAMESPACE(poly_sub)
+void poly_sub(poly *c, const poly *a, const poly *b);
+#define poly_shiftl DILITHIUM_NAMESPACE(poly_shiftl)
+void poly_shiftl(poly *a);
+
+#define poly_ntt DILITHIUM_NAMESPACE(poly_ntt)
+void poly_ntt(poly *a);
+#define poly_invntt_tomont DILITHIUM_NAMESPACE(poly_invntt_tomont)
+void poly_invntt_tomont(poly *a);
+#define poly_pointwise_montgomery DILITHIUM_NAMESPACE(poly_pointwise_montgomery)
+void poly_pointwise_montgomery(poly *c, const poly *a, const poly *b);
+
+#define poly_power2round DILITHIUM_NAMESPACE(poly_power2round)
+void poly_power2round(poly *a1, poly *a0, const poly *a);
+#define poly_decompose DILITHIUM_NAMESPACE(poly_decompose)
+void poly_decompose(poly *a1, poly *a0, const poly *a);
+#define poly_make_hint DILITHIUM_NAMESPACE(poly_make_hint)
+unsigned int poly_make_hint(poly *h, const poly *a0, const poly *a1);
+#define poly_use_hint DILITHIUM_NAMESPACE(poly_use_hint)
+void poly_use_hint(poly *b, const poly *a, const poly *h);
+
+#define poly_chknorm DILITHIUM_NAMESPACE(poly_chknorm)
+int poly_chknorm(const poly *a, int32_t B);
+#define poly_uniform DILITHIUM_NAMESPACE(poly_uniform)
+void poly_uniform(poly *a,
+                  const uint8_t seed[SEEDBYTES],
+                  uint16_t nonce);
+#define poly_uniformx2 DILITHIUM_NAMESPACE(poly_uniformx2)
+void poly_uniformx2(poly *a0, poly *a1,
+                    const uint8_t seed[SEEDBYTES],
+                    uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_eta DILITHIUM_NAMESPACE(poly_uniform_eta)
+void poly_uniform_eta(poly *a,
+                      const uint8_t seed[CRHBYTES],
+                      uint16_t nonce);
+#define poly_uniform_etax2 DILITHIUM_NAMESPACE(poly_uniform_etax2)
+void poly_uniform_etax2(poly *a0, poly *a1,
+                        const uint8_t seed[CRHBYTES],
+                        uint16_t nonce0, uint16_t nonce1);
+#define poly_uniform_gamma1 DILITHIUM_NAMESPACE(poly_uniform_gamma1)
+void poly_uniform_gamma1(poly *a,
+                         const uint8_t seed[CRHBYTES],
+                         uint16_t nonce);
+#define poly_uniform_gamma1x2 DILITHIUM_NAMESPACE(poly_uniform_gamma1x2)
+void poly_uniform_gamma1x2(poly *a0, poly *a1,
+                           const uint8_t seed[CRHBYTES],
+                           uint16_t nonce0, uint16_t nonce1);
+#define poly_challenge DILITHIUM_NAMESPACE(poly_challenge)
+void poly_challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define polyeta_pack DILITHIUM_NAMESPACE(polyeta_pack)
+void polyeta_pack(uint8_t *r, const poly *a);
+#define polyeta_unpack DILITHIUM_NAMESPACE(polyeta_unpack)
+void polyeta_unpack(poly *r, const uint8_t *a);
+
+#define polyt1_pack DILITHIUM_NAMESPACE(polyt1_pack)
+void polyt1_pack(uint8_t *r, const poly *a);
+#define polyt1_unpack DILITHIUM_NAMESPACE(polyt1_unpack)
+void polyt1_unpack(poly *r, const uint8_t *a);
+
+#define polyt0_pack DILITHIUM_NAMESPACE(polyt0_pack)
+void polyt0_pack(uint8_t *r, const poly *a);
+#define polyt0_unpack DILITHIUM_NAMESPACE(polyt0_unpack)
+void polyt0_unpack(poly *r, const uint8_t *a);
+
+#define polyz_pack DILITHIUM_NAMESPACE(polyz_pack)
+void polyz_pack(uint8_t *r, const poly *a);
+#define polyz_unpack DILITHIUM_NAMESPACE(polyz_unpack)
+void polyz_unpack(poly *r, const uint8_t *a);
+
+#define polyw1_pack DILITHIUM_NAMESPACE(polyw1_pack)
+void polyw1_pack(uint8_t *r, const poly *a);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/polyvec.c

@@ -0,0 +1,452 @@
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stdint.h>
+
+#include "reduce.h"
+
+static const int32_t l_montgomery_const[4] = {
+    DILITHIUM_Q, DILITHIUM_QINV
+};
+
+/*************************************************
+* Name:        expand_mat
+*
+* Description: Implementation of ExpandA. Generates matrix A with uniformly
+*              random coefficients a_{i,j} by performing rejection
+*              sampling on the output stream of SHAKE128(rho|j|i)
+*              or AES256CTR(rho,j|i).
+*
+* Arguments:   - polyvecl mat[K]: output matrix
+*              - const uint8_t rho[]: byte array containing seed rho
+**************************************************/
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]) {
+    unsigned int i, j;
+
+    for (j = 0; j < L; ++j) {
+        for (i = 0; i < K; i += 2) {
+            poly_uniformx2(&mat[i + 0].vec[j], &mat[i + 1].vec[j], rho, (uint16_t) ((i << 8) + j), (uint16_t) (((i + 1) << 8) + j));
+        }
+    }
+}
+
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyvecl_pointwise_acc_montgomery(&t->vec[i], &mat[i], v);
+    }
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length L **************/
+/**************************************************************/
+
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+}
+
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < L - 1; i += 2) {
+        poly_uniform_gamma1x2(&v->vec[i + 0], &v->vec[i + 1], seed, (uint16_t) (L * nonce + i + 0), (uint16_t) (L * nonce + i + 1));
+    }
+    if (L & 1) {
+        poly_uniform_gamma1(&v->vec[i], seed, (uint16_t) (L * nonce + L - 1));
+    }
+}
+
+void polyvecl_reduce(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length L
+*              to standard representatives.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_freeze(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_add
+*
+* Description: Add vectors of polynomials of length L.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyvecl *w: pointer to output vector
+*              - const polyvecl *u: pointer to first summand
+*              - const polyvecl *v: pointer to second summand
+**************************************************/
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length L. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyvecl *v: pointer to input/output vector
+**************************************************/
+void polyvecl_ntt(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+void polyvecl_invntt_tomont(polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v) {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyvecl_pointwise_acc_montgomery
+*
+* Description: Pointwise multiply vectors of polynomials of length L, multiply
+*              resulting vector by 2^{-32} and add (accumulate) polynomials
+*              in it. Input/output vectors are in NTT domain representation.
+*
+* Arguments:   - poly *w: output polynomial
+*              - const polyvecl *u: pointer to first input vector
+*              - const polyvecl *v: pointer to second input vector
+**************************************************/
+extern void PQCLEAN_DILITHIUM5_AARCH64_asm_polyvecl_pointwise_acc_montgomery(int32_t *, const int32_t *, const int32_t *, const int32_t *);
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v) {
+    PQCLEAN_DILITHIUM5_AARCH64_asm_polyvecl_pointwise_acc_montgomery(w->coeffs, u->vec[0].coeffs, v->vec[0].coeffs, l_montgomery_const);
+}
+
+/*************************************************
+* Name:        polyvecl_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length L.
+*              Assumes input polyvecl to be reduced by polyvecl_reduce().
+*
+* Arguments:   - const polyvecl *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials is strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyvecl_chknorm(const polyvecl *v, int32_t bound)  {
+    unsigned int i;
+
+    for (i = 0; i < L; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/**************************************************************/
+/************ Vectors of polynomials of length K **************/
+/**************************************************************/
+
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_uniform_eta(&v->vec[i], seed, nonce++);
+    }
+
+}
+
+/*************************************************
+* Name:        polyveck_reduce
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to representatives in [-6283009,6283007].
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_reduce(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_reduce(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_caddq
+*
+* Description: For all coefficients of polynomials in vector of length K
+*              add Q if coefficient is negative.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_caddq(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_caddq(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_freeze
+*
+* Description: Reduce coefficients of polynomials in vector of length K
+*              to standard representatives.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_freeze(polyveck *v)  {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_freeze(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_add
+*
+* Description: Add vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first summand
+*              - const polyveck *v: pointer to second summand
+**************************************************/
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_add(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_sub
+*
+* Description: Subtract vectors of polynomials of length K.
+*              No modular reduction is performed.
+*
+* Arguments:   - polyveck *w: pointer to output vector
+*              - const polyveck *u: pointer to first input vector
+*              - const polyveck *v: pointer to second input vector to be
+*                                   subtracted from first input vector
+**************************************************/
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_sub(&w->vec[i], &u->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_shiftl
+*
+* Description: Multiply vector of polynomials of Length K by 2^D without modular
+*              reduction. Assumes input coefficients to be less than 2^{31-D}.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_shiftl(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_shiftl(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_ntt
+*
+* Description: Forward NTT of all polynomials in vector of length K. Output
+*              coefficients can be up to 16*Q larger than input coefficients.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_ntt(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_ntt(&v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_invntt_tomont
+*
+* Description: Inverse NTT and multiplication by 2^{32} of polynomials
+*              in vector of length K. Input coefficients need to be less
+*              than 2*Q.
+*
+* Arguments:   - polyveck *v: pointer to input/output vector
+**************************************************/
+void polyveck_invntt_tomont(polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_invntt_tomont(&v->vec[i]);
+    }
+}
+
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_pointwise_montgomery(&r->vec[i], a, &v->vec[i]);
+    }
+}
+
+
+/*************************************************
+* Name:        polyveck_chknorm
+*
+* Description: Check infinity norm of polynomials in vector of length K.
+*              Assumes input polyveck to be reduced by polyveck_reduce().
+*
+* Arguments:   - const polyveck *v: pointer to vector
+*              - int32_t B: norm bound
+*
+* Returns 0 if norm of all polynomials are strictly smaller than B <= (Q-1)/8
+* and 1 otherwise.
+**************************************************/
+int polyveck_chknorm(const polyveck *v, int32_t bound) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        if (poly_chknorm(&v->vec[i], bound)) {
+            return 1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        polyveck_power2round
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute a0, a1 such that a mod^+ Q = a1*2^D + a0
+*              with -2^{D-1} < a0 <= 2^{D-1}. Assumes coefficients to be
+*              standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_power2round(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_decompose
+*
+* Description: For all coefficients a of polynomials in vector of length K,
+*              compute high and low bits a0, a1 such a mod^+ Q = a1*ALPHA + a0
+*              with -ALPHA/2 < a0 <= ALPHA/2 except a1 = (Q-1)/ALPHA where we
+*              set a1 = 0 and -ALPHA/2 <= a0 = a mod Q - Q < 0.
+*              Assumes coefficients to be standard representatives.
+*
+* Arguments:   - polyveck *v1: pointer to output vector of polynomials with
+*                              coefficients a1
+*              - polyveck *v0: pointer to output vector of polynomials with
+*                              coefficients a0
+*              - const polyveck *v: pointer to input vector
+**************************************************/
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_decompose(&v1->vec[i], &v0->vec[i], &v->vec[i]);
+    }
+}
+
+/*************************************************
+* Name:        polyveck_make_hint
+*
+* Description: Compute hint vector.
+*
+* Arguments:   - polyveck *h: pointer to output vector
+*              - const polyveck *v0: pointer to low part of input vector
+*              - const polyveck *v1: pointer to high part of input vector
+*
+* Returns number of 1 bits.
+**************************************************/
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1) {
+    unsigned int i, s = 0;
+
+    for (i = 0; i < K; ++i) {
+        s += poly_make_hint(&h->vec[i], &v0->vec[i], &v1->vec[i]);
+    }
+
+    return s;
+}
+
+/*************************************************
+* Name:        polyveck_use_hint
+*
+* Description: Use hint vector to correct the high bits of input vector.
+*
+* Arguments:   - polyveck *w: pointer to output vector of polynomials with
+*                             corrected high bits
+*              - const polyveck *u: pointer to input vector
+*              - const polyveck *h: pointer to input hint vector
+**************************************************/
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        poly_use_hint(&w->vec[i], &u->vec[i], &h->vec[i]);
+    }
+}
+
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1) {
+    unsigned int i;
+
+    for (i = 0; i < K; ++i) {
+        polyw1_pack(&r[i * POLYW1_PACKEDBYTES], &w1->vec[i]);
+    }
+}

src/sign/dilithium/dilithium5/aarch64/polyvec.h

@@ -0,0 +1,96 @@
+#ifndef POLYVEC_H
+#define POLYVEC_H
+#include "params.h"
+#include "poly.h"
+#include <stdint.h>
+
+/* Vectors of polynomials of length L */
+typedef struct {
+    poly vec[L];
+} polyvecl;
+
+#define polyvecl_uniform_eta DILITHIUM_NAMESPACE(polyvecl_uniform_eta)
+void polyvecl_uniform_eta(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_uniform_gamma1 DILITHIUM_NAMESPACE(polyvecl_uniform_gamma1)
+void polyvecl_uniform_gamma1(polyvecl *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyvecl_reduce DILITHIUM_NAMESPACE(polyvecl_reduce)
+void polyvecl_reduce(polyvecl *v);
+
+#define polyvecl_freeze DILITHIUM_NAMESPACE(polyvecl_freeze)
+void polyvecl_freeze(polyvecl *v);
+
+#define polyvecl_add DILITHIUM_NAMESPACE(polyvecl_add)
+void polyvecl_add(polyvecl *w, const polyvecl *u, const polyvecl *v);
+
+#define polyvecl_ntt DILITHIUM_NAMESPACE(polyvecl_ntt)
+void polyvecl_ntt(polyvecl *v);
+#define polyvecl_invntt_tomont DILITHIUM_NAMESPACE(polyvecl_invntt_tomont)
+void polyvecl_invntt_tomont(polyvecl *v);
+#define polyvecl_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_poly_montgomery)
+void polyvecl_pointwise_poly_montgomery(polyvecl *r, const poly *a, const polyvecl *v);
+#define polyvecl_pointwise_acc_montgomery DILITHIUM_NAMESPACE(polyvecl_pointwise_acc_montgomery)
+void polyvecl_pointwise_acc_montgomery(poly *w,
+                                       const polyvecl *u,
+                                       const polyvecl *v);
+
+
+#define polyvecl_chknorm DILITHIUM_NAMESPACE(polyvecl_chknorm)
+int polyvecl_chknorm(const polyvecl *v, int32_t B);
+
+
+
+/* Vectors of polynomials of length K */
+typedef struct {
+    poly vec[K];
+} polyveck;
+
+#define polyveck_uniform_eta DILITHIUM_NAMESPACE(polyveck_uniform_eta)
+void polyveck_uniform_eta(polyveck *v, const uint8_t seed[CRHBYTES], uint16_t nonce);
+
+#define polyveck_reduce DILITHIUM_NAMESPACE(polyveck_reduce)
+void polyveck_reduce(polyveck *v);
+#define polyveck_caddq DILITHIUM_NAMESPACE(polyveck_caddq)
+void polyveck_caddq(polyveck *v);
+#define polyveck_freeze DILITHIUM_NAMESPACE(polyveck_freeze)
+void polyveck_freeze(polyveck *v);
+
+#define polyveck_add DILITHIUM_NAMESPACE(polyveck_add)
+void polyveck_add(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_sub DILITHIUM_NAMESPACE(polyveck_sub)
+void polyveck_sub(polyveck *w, const polyveck *u, const polyveck *v);
+#define polyveck_shiftl DILITHIUM_NAMESPACE(polyveck_shiftl)
+void polyveck_shiftl(polyveck *v);
+
+#define polyveck_ntt DILITHIUM_NAMESPACE(polyveck_ntt)
+void polyveck_ntt(polyveck *v);
+#define polyveck_invntt_tomont DILITHIUM_NAMESPACE(polyveck_invntt_tomont)
+void polyveck_invntt_tomont(polyveck *v);
+#define polyveck_pointwise_poly_montgomery DILITHIUM_NAMESPACE(polyveck_pointwise_poly_montgomery)
+void polyveck_pointwise_poly_montgomery(polyveck *r, const poly *a, const polyveck *v);
+
+#define polyveck_chknorm DILITHIUM_NAMESPACE(polyveck_chknorm)
+int polyveck_chknorm(const polyveck *v, int32_t B);
+
+#define polyveck_power2round DILITHIUM_NAMESPACE(polyveck_power2round)
+void polyveck_power2round(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_decompose DILITHIUM_NAMESPACE(polyveck_decompose)
+void polyveck_decompose(polyveck *v1, polyveck *v0, const polyveck *v);
+#define polyveck_make_hint DILITHIUM_NAMESPACE(polyveck_make_hint)
+unsigned int polyveck_make_hint(polyveck *h,
+                                const polyveck *v0,
+                                const polyveck *v1);
+#define polyveck_use_hint DILITHIUM_NAMESPACE(polyveck_use_hint)
+void polyveck_use_hint(polyveck *w, const polyveck *u, const polyveck *h);
+
+#define polyveck_pack_w1 DILITHIUM_NAMESPACE(polyveck_pack_w1)
+void polyveck_pack_w1(uint8_t r[K * POLYW1_PACKEDBYTES], const polyveck *w1);
+
+#define polyvec_matrix_expand DILITHIUM_NAMESPACE(polyvec_matrix_expand)
+void polyvec_matrix_expand(polyvecl mat[K], const uint8_t rho[SEEDBYTES]);
+
+#define polyvec_matrix_pointwise_montgomery DILITHIUM_NAMESPACE(polyvec_matrix_pointwise_montgomery)
+void polyvec_matrix_pointwise_montgomery(polyveck *t, const polyvecl mat[K], const polyvecl *v);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/reduce.c

@@ -0,0 +1,69 @@
+#include "params.h"
+#include "reduce.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        montgomery_reduce
+*
+* Description: For finite field element a with -2^{31}Q <= a <= Q*2^31,
+*              compute r \equiv a*2^{-32} (mod Q) such that -Q < r < Q.
+*
+* Arguments:   - int64_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t montgomery_reduce(int64_t a) {
+    int32_t t;
+
+    t = (int32_t)((uint64_t)a * (uint64_t)DILITHIUM_QINV);
+    t = (a - (int64_t)t * DILITHIUM_Q) >> 32;
+    return t;
+}
+
+/*************************************************
+* Name:        reduce32
+*
+* Description: For finite field element a with a <= 2^{31} - 2^{22} - 1,
+*              compute r \equiv a (mod Q) such that -6283009 <= r <= 6283007.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t reduce32(int32_t a) {
+    int32_t t;
+
+    t = (a + (1 << 22)) >> 23;
+    t = a - t * DILITHIUM_Q;
+    return t;
+}
+
+/*************************************************
+* Name:        caddq
+*
+* Description: Add Q if input coefficient is negative.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t caddq(int32_t a) {
+    a += (a >> 31) & DILITHIUM_Q;
+    return a;
+}
+
+/*************************************************
+* Name:        freeze
+*
+* Description: For finite field element a, compute standard
+*              representative r = a mod^+ Q.
+*
+* Arguments:   - int32_t: finite field element a
+*
+* Returns r.
+**************************************************/
+int32_t freeze(int32_t a) {
+    a = reduce32(a);
+    a = caddq(a);
+    return a;
+}

src/sign/dilithium/dilithium5/aarch64/reduce.h

@@ -0,0 +1,20 @@
+#ifndef REDUCE_H
+#define REDUCE_H
+#include "params.h"
+#include <stdint.h>
+
+#define DILITHIUM_QINV 58728449 // q^(-1) mod 2^32
+
+#define montgomery_reduce DILITHIUM_NAMESPACE(montgomery_reduce)
+int32_t montgomery_reduce(int64_t a);
+
+#define reduce32 DILITHIUM_NAMESPACE(reduce32)
+int32_t reduce32(int32_t a);
+
+#define caddq DILITHIUM_NAMESPACE(caddq)
+int32_t caddq(int32_t a);
+
+#define freeze DILITHIUM_NAMESPACE(freeze)
+int32_t freeze(int32_t a);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/rounding.c

@@ -0,0 +1,96 @@
+#include "params.h"
+#include "rounding.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        power2round
+*
+* Description: For finite field element a, compute a0, a1 such that
+*              a mod^+ Q = a1*2^D + a0 with -2^{D-1} < a0 <= 2^{D-1}.
+*              Assumes a to be standard representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t power2round(int32_t *a0, int32_t a)  {
+    int32_t a1;
+
+    a1 = (a + (1 << (D - 1)) - 1) >> D;
+    *a0 = a - (a1 << D);
+    return a1;
+}
+
+/*************************************************
+* Name:        decompose
+*
+* Description: For finite field element a, compute high and low bits a0, a1 such
+*              that a mod^+ Q = a1*ALPHA + a0 with -ALPHA/2 < a0 <= ALPHA/2 except
+*              if a1 = (Q-1)/ALPHA where we set a1 = 0 and
+*              -ALPHA/2 <= a0 = a mod^+ Q - Q < 0. Assumes a to be standard
+*              representative.
+*
+* Arguments:   - int32_t a: input element
+*              - int32_t *a0: pointer to output element a0
+*
+* Returns a1.
+**************************************************/
+int32_t decompose(int32_t *a0, int32_t a) {
+    int32_t a1;
+
+    a1  = (a + 127) >> 7;
+
+    a1  = (a1 * 1025 + (1 << 21)) >> 22;
+    a1 &= 15;
+
+
+    *a0  = a - a1 * 2 * GAMMA2;
+    *a0 -= (((DILITHIUM_Q - 1) / 2 - *a0) >> 31) & DILITHIUM_Q;
+    return a1;
+}
+
+/*************************************************
+* Name:        make_hint
+*
+* Description: Compute hint bit indicating whether the low bits of the
+*              input element overflow into the high bits.
+*
+* Arguments:   - int32_t a0: low bits of input element
+*              - int32_t a1: high bits of input element
+*
+* Returns 1 if overflow.
+**************************************************/
+unsigned int make_hint(int32_t a0, int32_t a1) {
+    if (a0 > GAMMA2 || a0 < -GAMMA2 || (a0 == -GAMMA2 && a1 != 0)) {
+        return 1;
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        use_hint
+*
+* Description: Correct high bits according to hint.
+*
+* Arguments:   - int32_t a: input element
+*              - unsigned int hint: hint bit
+*
+* Returns corrected high bits.
+**************************************************/
+int32_t use_hint(int32_t a, unsigned int hint) {
+    int32_t a0, a1;
+
+    a1 = decompose(&a0, a);
+    if (hint == 0) {
+        return a1;
+    }
+
+
+    if (a0 > 0) {
+        return (a1 + 1) & 15;
+    }
+    return (a1 - 1) & 15;
+
+}

src/sign/dilithium/dilithium5/aarch64/rounding.h

@@ -0,0 +1,18 @@
+#ifndef ROUNDING_H
+#define ROUNDING_H
+#include "params.h"
+#include <stdint.h>
+
+#define power2round DILITHIUM_NAMESPACE(power2round)
+int32_t power2round(int32_t *a0, int32_t a);
+
+#define decompose DILITHIUM_NAMESPACE(decompose)
+int32_t decompose(int32_t *a0, int32_t a);
+
+#define make_hint DILITHIUM_NAMESPACE(make_hint)
+unsigned int make_hint(int32_t a0, int32_t a1);
+
+#define use_hint DILITHIUM_NAMESPACE(use_hint)
+int32_t use_hint(int32_t a, unsigned int hint);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/sign.c

@@ -0,0 +1,343 @@
+#include "fips202.h"
+#include "packing.h"
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include "randombytes.h"
+#include "sign.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+/*************************************************
+* Name:        crypto_sign_keypair
+*
+* Description: Generates public and private key.
+*
+* Arguments:   - uint8_t *pk: pointer to output public key (allocated
+*                             array of CRYPTO_PUBLICKEYBYTES bytes)
+*              - uint8_t *sk: pointer to output private key (allocated
+*                             array of CRYPTO_SECRETKEYBYTES bytes)
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk) {
+    uint8_t seedbuf[2 * SEEDBYTES + CRHBYTES];
+    uint8_t tr[SEEDBYTES];
+    const uint8_t *rho, *rhoprime, *key;
+    polyvecl mat[K];
+    polyvecl s1, s1hat;
+    polyveck s2, t1, t0;
+
+    /* Get randomness for rho, rhoprime and key */
+    randombytes(seedbuf, SEEDBYTES);
+    shake256(seedbuf, 2 * SEEDBYTES + CRHBYTES, seedbuf, SEEDBYTES);
+    rho = seedbuf;
+    rhoprime = rho + SEEDBYTES;
+    key = rhoprime + CRHBYTES;
+
+    /* Expand matrix */
+    polyvec_matrix_expand(mat, rho);
+
+    /* Sample short vectors s1 and s2 */
+    polyvecl_uniform_eta(&s1, rhoprime, 0);
+    polyveck_uniform_eta(&s2, rhoprime, L);
+
+    /* Matrix-vector multiplication */
+    s1hat = s1;
+    polyvecl_ntt(&s1hat);
+    polyvec_matrix_pointwise_montgomery(&t1, mat, &s1hat);
+    polyveck_reduce(&t1);
+    polyveck_invntt_tomont(&t1);
+
+    /* Add error vector s2 */
+    polyveck_add(&t1, &t1, &s2);
+
+    /* Extract t1 and write public key */
+    polyveck_caddq(&t1);
+    polyveck_power2round(&t1, &t0, &t1);
+    pack_pk(pk, rho, &t1);
+
+    /* Compute H(rho, t1) and write secret key */
+    shake256(tr, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    pack_sk(sk, rho, tr, key, &t0, &s1, &s2);
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_signature
+*
+* Description: Computes signature.
+*
+* Arguments:   - uint8_t *sig:   pointer to output signature (of length CRYPTO_BYTES)
+*              - size_t *siglen: pointer to output length of signature
+*              - uint8_t *m:     pointer to message to be signed
+*              - size_t mlen:    length of message
+*              - uint8_t *sk:    pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign_signature(uint8_t *sig,
+                          size_t *siglen,
+                          const uint8_t *m,
+                          size_t mlen,
+                          const uint8_t *sk) {
+    unsigned int n;
+    uint8_t seedbuf[3 * SEEDBYTES + 2 * CRHBYTES];
+    uint8_t *rho, *tr, *key, *mu, *rhoprime;
+    uint16_t nonce = 0;
+    polyvecl mat[K], s1, y, z;
+    polyveck t0, s2, w1, w0, h;
+    poly cp;
+    shake256incctx state;
+
+    rho = seedbuf;
+    tr = rho + SEEDBYTES;
+    key = tr + SEEDBYTES;
+    mu = key + SEEDBYTES;
+    rhoprime = mu + CRHBYTES;
+    unpack_sk(rho, tr, key, &t0, &s1, &s2, sk);
+
+    /* Compute CRH(tr, msg) */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, tr, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    shake256(rhoprime, CRHBYTES, key, SEEDBYTES + CRHBYTES);
+
+    /* Expand matrix and transform vectors */
+    polyvec_matrix_expand(mat, rho);
+    polyvecl_ntt(&s1);
+    polyveck_ntt(&s2);
+    polyveck_ntt(&t0);
+
+rej:
+    /* Sample intermediate vector y */
+    polyvecl_uniform_gamma1(&y, rhoprime, nonce++);
+
+    /* Matrix-vector multiplication */
+    z = y;
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Decompose w and call the random oracle */
+    polyveck_caddq(&w1);
+    polyveck_decompose(&w1, &w0, &w1);
+    polyveck_pack_w1(sig, &w1);
+
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, sig, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(sig, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    poly_challenge(&cp, sig);
+    poly_ntt(&cp);
+
+    /* Compute z, reject if it reveals secret */
+    polyvecl_pointwise_poly_montgomery(&z, &cp, &s1);
+    polyvecl_invntt_tomont(&z);
+    polyvecl_add(&z, &z, &y);
+    polyvecl_reduce(&z);
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        goto rej;
+    }
+
+    /* Check that subtracting cs2 does not change high bits of w and low bits
+     * do not reveal secret information */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &s2);
+    polyveck_invntt_tomont(&h);
+    polyveck_sub(&w0, &w0, &h);
+    polyveck_reduce(&w0);
+    if (polyveck_chknorm(&w0, GAMMA2 - BETA)) {
+        goto rej;
+    }
+
+    /* Compute hints for w1 */
+    polyveck_pointwise_poly_montgomery(&h, &cp, &t0);
+    polyveck_invntt_tomont(&h);
+    polyveck_reduce(&h);
+    if (polyveck_chknorm(&h, GAMMA2)) {
+        goto rej;
+    }
+
+    polyveck_add(&w0, &w0, &h);
+    n = polyveck_make_hint(&h, &w0, &w1);
+    if (n > OMEGA) {
+        goto rej;
+    }
+
+    /* Write signature */
+    pack_sig(sig, sig, &z, &h);
+    *siglen = CRYPTO_BYTES;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign
+*
+* Description: Compute signed message.
+*
+* Arguments:   - uint8_t *sm: pointer to output signed message (allocated
+*                             array with CRYPTO_BYTES + mlen bytes),
+*                             can be equal to m
+*              - size_t *smlen: pointer to output length of signed
+*                               message
+*              - const uint8_t *m: pointer to message to be signed
+*              - size_t mlen: length of message
+*              - const uint8_t *sk: pointer to bit-packed secret key
+*
+* Returns 0 (success)
+**************************************************/
+int crypto_sign(uint8_t *sm,
+                size_t *smlen,
+                const uint8_t *m,
+                size_t mlen,
+                const uint8_t *sk) {
+    size_t i;
+
+    for (i = 0; i < mlen; ++i) {
+        sm[CRYPTO_BYTES + mlen - 1 - i] = m[mlen - 1 - i];
+    }
+    crypto_sign_signature(sm, smlen, sm + CRYPTO_BYTES, mlen, sk);
+    *smlen += mlen;
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_verify
+*
+* Description: Verifies signature.
+*
+* Arguments:   - uint8_t *m: pointer to input signature
+*              - size_t siglen: length of signature
+*              - const uint8_t *m: pointer to message
+*              - size_t mlen: length of message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signature could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_verify(const uint8_t *sig,
+                       size_t siglen,
+                       const uint8_t *m,
+                       size_t mlen,
+                       const uint8_t *pk) {
+    unsigned int i;
+    uint8_t buf[K * POLYW1_PACKEDBYTES];
+    uint8_t rho[SEEDBYTES];
+    uint8_t mu[CRHBYTES];
+    uint8_t c[SEEDBYTES];
+    uint8_t c2[SEEDBYTES];
+    poly cp;
+    polyvecl mat[K], z;
+    polyveck t1, w1, h;
+    shake256incctx state;
+
+    if (siglen != CRYPTO_BYTES) {
+        return -1;
+    }
+
+    unpack_pk(rho, &t1, pk);
+    if (unpack_sig(c, &z, &h, sig)) {
+        return -1;
+    }
+    if (polyvecl_chknorm(&z, GAMMA1 - BETA)) {
+        return -1;
+    }
+
+    /* Compute CRH(H(rho, t1), msg) */
+    shake256(mu, SEEDBYTES, pk, CRYPTO_PUBLICKEYBYTES);
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, SEEDBYTES);
+    shake256_inc_absorb(&state, m, mlen);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(mu, CRHBYTES, &state);
+    shake256_inc_ctx_release(&state);
+
+    /* Matrix-vector multiplication; compute Az - c2^dt1 */
+    poly_challenge(&cp, c);
+    polyvec_matrix_expand(mat, rho);
+
+    polyvecl_ntt(&z);
+    polyvec_matrix_pointwise_montgomery(&w1, mat, &z);
+
+    poly_ntt(&cp);
+    polyveck_shiftl(&t1);
+    polyveck_ntt(&t1);
+    polyveck_pointwise_poly_montgomery(&t1, &cp, &t1);
+
+    polyveck_sub(&w1, &w1, &t1);
+    polyveck_reduce(&w1);
+    polyveck_invntt_tomont(&w1);
+
+    /* Reconstruct w1 */
+    polyveck_caddq(&w1);
+    polyveck_use_hint(&w1, &w1, &h);
+    polyveck_pack_w1(buf, &w1);
+
+    /* Call random oracle and verify challenge */
+    shake256_inc_init(&state);
+    shake256_inc_absorb(&state, mu, CRHBYTES);
+    shake256_inc_absorb(&state, buf, K * POLYW1_PACKEDBYTES);
+    shake256_inc_finalize(&state);
+    shake256_inc_squeeze(c2, SEEDBYTES, &state);
+    shake256_inc_ctx_release(&state);
+    for (i = 0; i < SEEDBYTES; ++i) {
+        if (c[i] != c2[i]) {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+/*************************************************
+* Name:        crypto_sign_open
+*
+* Description: Verify signed message.
+*
+* Arguments:   - uint8_t *m: pointer to output message (allocated
+*                            array with smlen bytes), can be equal to sm
+*              - size_t *mlen: pointer to output length of message
+*              - const uint8_t *sm: pointer to signed message
+*              - size_t smlen: length of signed message
+*              - const uint8_t *pk: pointer to bit-packed public key
+*
+* Returns 0 if signed message could be verified correctly and -1 otherwise
+**************************************************/
+int crypto_sign_open(uint8_t *m,
+                     size_t *mlen,
+                     const uint8_t *sm,
+                     size_t smlen,
+                     const uint8_t *pk) {
+    size_t i;
+
+    if (smlen < CRYPTO_BYTES) {
+        goto badsig;
+    }
+
+    *mlen = smlen - CRYPTO_BYTES;
+    if (crypto_sign_verify(sm, CRYPTO_BYTES, sm + CRYPTO_BYTES, *mlen, pk)) {
+        goto badsig;
+    } else {
+        /* All good, copy msg, return 0 */
+        for (i = 0; i < *mlen; ++i) {
+            m[i] = sm[CRYPTO_BYTES + i];
+        }
+        return 0;
+    }
+
+badsig:
+    /* Signature verification failed */
+    *mlen = (size_t) -1;
+    for (i = 0; i < smlen; ++i) {
+        m[i] = 0;
+    }
+
+    return -1;
+}

src/sign/dilithium/dilithium5/aarch64/sign.h

@@ -0,0 +1,37 @@
+#ifndef SIGN_H
+#define SIGN_H
+#include "params.h"
+#include "poly.h"
+#include "polyvec.h"
+#include <stddef.h>
+#include <stdint.h>
+
+
+
+#define challenge DILITHIUM_NAMESPACE(challenge)
+void challenge(poly *c, const uint8_t seed[SEEDBYTES]);
+
+#define crypto_sign_keypair DILITHIUM_NAMESPACE(crypto_sign_keypair)
+int crypto_sign_keypair(uint8_t *pk, uint8_t *sk);
+
+#define crypto_sign_signature DILITHIUM_NAMESPACE(crypto_sign_signature)
+int crypto_sign_signature(uint8_t *sig, size_t *siglen,
+                          const uint8_t *m, size_t mlen,
+                          const uint8_t *sk);
+
+#define crypto_sign DILITHIUM_NAMESPACETOP
+int crypto_sign(uint8_t *sm, size_t *smlen,
+                const uint8_t *m, size_t mlen,
+                const uint8_t *sk);
+
+#define crypto_sign_verify DILITHIUM_NAMESPACE(crypto_sign_verify)
+int crypto_sign_verify(const uint8_t *sig, size_t siglen,
+                       const uint8_t *m, size_t mlen,
+                       const uint8_t *pk);
+
+#define crypto_sign_open DILITHIUM_NAMESPACE(crypto_sign_open)
+int crypto_sign_open(uint8_t *m, size_t *mlen,
+                     const uint8_t *sm, size_t smlen,
+                     const uint8_t *pk);
+
+#endif

src/sign/dilithium/dilithium5/aarch64/symmetric-shake.c

@@ -0,0 +1,66 @@
+#include "fips202.h"
+#include "params.h"
+#include "symmetric.h"
+#include <stdint.h>
+
+void dilithium_shake128_stream_init(shake128incctx *state, const uint8_t seed[SEEDBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake128_inc_init(state);
+    shake128_inc_absorb(state, seed, SEEDBYTES);
+    shake128_inc_absorb(state, t, 2);
+    shake128_inc_finalize(state);
+}
+
+void dilithium_shake256_stream_init(shake256incctx *state, const uint8_t seed[CRHBYTES], uint16_t nonce) {
+    uint8_t t[2];
+    t[0] = (uint8_t) nonce;
+    t[1] = (uint8_t) (nonce >> 8);
+
+    shake256_inc_init(state);
+    shake256_inc_absorb(state, seed, CRHBYTES);
+    shake256_inc_absorb(state, t, 2);
+    shake256_inc_finalize(state);
+}
+
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[SEEDBYTES + 2 + 14];
+    uint8_t extseed2[SEEDBYTES + 2 + 14];
+
+    for (i = 0; i < SEEDBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[SEEDBYTES] = (uint8_t) nonce1;
+    extseed1[SEEDBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[SEEDBYTES  ] = (uint8_t) nonce2;
+    extseed2[SEEDBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake128x2_absorb(state, extseed1, extseed2, SEEDBYTES + 2);
+}
+
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2) {
+    unsigned int i;
+    uint8_t extseed1[CRHBYTES + 2 + 14];
+    uint8_t extseed2[CRHBYTES + 2 + 14];
+
+    for (i = 0; i < CRHBYTES; i++) {
+        extseed1[i] = seed[i];
+        extseed2[i] = seed[i];
+    }
+    extseed1[CRHBYTES] = (uint8_t) nonce1;
+    extseed1[CRHBYTES + 1] = (uint8_t) (nonce1 >> 8);
+
+    extseed2[CRHBYTES  ] = (uint8_t) nonce2;
+    extseed2[CRHBYTES + 1] = (uint8_t) (nonce2 >> 8);
+
+    shake256x2_absorb(state, extseed1, extseed2, CRHBYTES + 2);
+}

src/sign/dilithium/dilithium5/aarch64/symmetric.h

@@ -0,0 +1,47 @@
+#ifndef SYMMETRIC_H
+#define SYMMETRIC_H
+#include "fips202.h"
+#include "fips202x2.h"
+#include "params.h"
+#include <stdint.h>
+
+
+typedef shake128incctx stream128_state;
+typedef shake256incctx stream256_state;
+
+#define dilithium_shake128_stream_init DILITHIUM_NAMESPACE(dilithium_shake128_stream_init)
+void dilithium_shake128_stream_init(shake128incctx *state,
+                                    const uint8_t seed[SEEDBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake256_stream_init DILITHIUM_NAMESPACE(dilithium_shake256_stream_init)
+void dilithium_shake256_stream_init(shake256incctx *state,
+                                    const uint8_t seed[CRHBYTES],
+                                    uint16_t nonce);
+
+#define dilithium_shake128x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake128x2_stream_init)
+void dilithium_shake128x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[SEEDBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+#define dilithium_shake256x2_stream_init DILITHIUM_NAMESPACE(dilithium_shake256x2_stream_init)
+void dilithium_shake256x2_stream_init(keccakx2_state *state,
+                                      const uint8_t seed[CRHBYTES],
+                                      uint16_t nonce1, uint16_t nonce2);
+
+
+#define STREAM128_BLOCKBYTES SHAKE128_RATE
+#define STREAM256_BLOCKBYTES SHAKE256_RATE
+
+#define stream128_init(STATE, SEED, NONCE) \
+    dilithium_shake128_stream_init(STATE, SEED, NONCE)
+#define stream128_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake128_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE128_RATE), STATE)
+#define stream128_release(STATE) shake128_inc_ctx_release(STATE)
+#define stream256_init(STATE, SEED, NONCE) \
+    dilithium_shake256_stream_init(STATE, SEED, NONCE)
+#define stream256_squeezeblocks(OUT, OUTBLOCKS, STATE) \
+    shake256_inc_squeeze(OUT, (OUTBLOCKS)*(SHAKE256_RATE), STATE)
+#define stream256_release(STATE) shake256_inc_ctx_release(STATE)
+
+
+#endif