Refactor provider helpers and fix parameter handling

[olszomal]

Pull Request Type

• Bug fix
• New feature
• Code style / formatting / renaming
• Refactoring (no functional or API changes)
• Build / CI related changes
• Documentation
• Other (please describe):

Related Issue

Issue number: N/A

Current Behavior

Provider helper functions are reorganized for clarity.

Initialization parameters are applied consistently, RSA-PSS salt length values are validated, and signature output length is initialized from the supplied buffer capacity.

The fips=yes property is removed, since FIPS approval depends on the PKCS#11 module, token, and operation parameters.

EC private keys without an associated public point may trigger a NULL dereference in affected OpenSSL 3.x versions when public parameters are queried.

The test suite may also modify LD_LIBRARY_PATH before resolving all OpenSSL paths, causing system tools such as pkg-config to load an incompatible libcrypto.

New Behavior

Provider helper functions are reorganized for clarity.

The fips=yes property is removed, since FIPS approval depends on the PKCS#11 module, token, and operation parameters.

EC public-key parameter extraction avoids the affected OpenSSL NULL dereference for private-only EC keys.

OpenSSL paths used by the test suite are resolved before modifying LD_LIBRARY_PATH.

Scope of Changes

• Reorganize provider helper functions.
• Apply initialization parameters for signature, asymmetric cipher, and key exchange operations.
• Validate numeric RSA-PSS salt length values.
• Initialize signature output length from the supplied buffer capacity.
• Remove fips=yes from provider algorithm definitions.
• Avoid EC public point NULL dereference on affected OpenSSL versions.
• Resolve OpenSSL paths before updating LD_LIBRARY_PATH in the test suite.
• Fix OpenSSL setup in the EC duplicate key test.

Testing

• Existing tests
• New tests added
• Manual testing

Additional Notes

License Declaration

• I hereby agree to license my contribution under the project's license.

[mtrojnar]

Behavioral changes worth mentioning in the PR description

Although primarily a reorganization, the final commit includes functional changes:

• src/provider_helpers.c:2938-2969, 3717-3745, and 4127-4149 now apply parameters supplied during signature, asymmetric-cipher, and key-exchange initialization; the previous helpers ignored some initialization parameters.
• src/provider_helpers.c:2450-2486 replaces permissive atoi()-style PSS salt-length parsing with validated numeric parsing.
• src/provider_helpers.c:3130, 3384, 3429, and 3444 initialize *siglen from the output capacity before calling the PKCS#11 signing layer.

These appear beneficial, but mean the change is not strictly behavior-neutral. They would be worth documenting and possibly also covering with focused tests.