perl5360delta - what is new for perl v5.36.0
This document describes differences between the 5.34.0 release and the 5.36.0 release.
use v5.36
As always, use v5.36 turns on the feature bundle for that version of Perl.
The 5.36 bundle enables the signatures feature. Introduced in Perl version 5.20.0, and modified several times since, the subroutine signatures feature is now no longer considered experimental. It is now considered a stable language feature and no longer prints a warning.
use v5.36;
sub add ($x, $y) {
return $x + $y;
}
Despite this, certain elements of signatured subroutines remain experimental; see below.
The 5.36 bundle enables the isa feature. Introduced in Perl version 5.32.0, this operator has remained unchanged since then. The operator is now considered a stable language feature. For more detail see "Class Instance Operator" in perlop.
The 5.36 bundle also disables the features indirect, and multidimensional. These will forbid, respectively: the use of "indirect" method calls (like $x = new Class;); the use of a list expression as a hash key to simulate sparse multidimensional arrays. The specifics of these changes can be found in feature, but the short version is: this is a bit like having more use strict turned on, disabling features that cause more trouble than they're worth.
Furthermore, use v5.36 will also enable warnings as if you'd written use warnings.
Finally, with this release, the experimental switch feature, present in every feature bundle since they were introduced in v5.10, has been removed from the v5.36 bundle. If you want to use it (against our advice), you'll have to enable it explicitly.
A new command-line flag, -g, is available. It is a simpler alias for -0777.
For more information, see "-g" in perlrun.
See https://www.unicode.org/versions/Unicode14.0.0/ for details.
Prior to this release, the regex sets feature (officially named "Extended Bracketed Character Classes") was considered experimental. Introduced in Perl version 5.18.0, and modified several times since, this is now considered a stable language feature and its use no longer prints a warning. See "Extended Bracketed Character Classes" in perlrecharclass.
Prior to this release, any form of variable length lookbehind was considered experimental. With this release the experimental status has been reduced to cover only lookbehind that contains capturing parenthesis. This is because it is not clear if
"aaz"=~/(?=z)(?<=(a|aa))/
should match and leave $1 equaling "a" or "aa". Currently it will match the longest possible alternative, "aa". While we are confident that the overall construct will now match only when it should, we are not confident that we will keep the current "longest match" behavior.
Floating-point exceptions are now delivered immediately, in the same way as other "fault"-like signals such as SIGSEGV. This means one has at least a chance to catch such a signal with a $SIG{FPE} handler, e.g. so that die can report the line in perl that triggered it.
The "true" and "false" boolean values, often accessed by constructions like !!0 and !!1, as well as being returned from many core functions and operators, now remember their boolean nature even through assignment into variables. The new function is_bool() in builtin can check whether a value has boolean nature.
This is likely to be useful when interoperating with other languages or data-type serialisation, among other places.
You can now iterate over multiple values at a time by specifying a list of lexicals within parentheses. For example,
for my ($key, $value) (%hash) { ... }
for my ($left, $right, $gripping) (@moties) { ... }
Prior to perl v5.36, attempting to specify a list after for my was a syntax error.
This feature is currently experimental and will cause a warning of category experimental::for_list. For more detail see "Compound Statements" in perlsyn. See also "builtin::indexed" in this document, which is a handy companion to n-at-a-time foreach.
A new core module builtin has been added, which provides documentation for new always-present functions that are built into the interpreter.
say "Reference type of arrays is ", builtin::reftype([]);
It also provides a lexical import mechanism for providing short name versions of these functions.
use builtin 'reftype'; say "Reference type of arrays is ", reftype([]);
This builtin function mechanism and the functions it provides are all currently experimental. We expect that builtin itself will cease to be experimental in the near future, but that individual functions in it may become stable on an ongoing basis. Other functions will be added to builtin over time.
For details, see builtin, but here's a summary of builtin functions in v5.36:
This function treats its argument as a string, returning the result of removing all white space at its beginning and ending.
This function returns a list twice as big as its argument list, where each item is preceded by its index within that list. This is primarily useful for using the new foreach syntax with multiple iterator variables to iterate over an array or list, while also tracking the index of each item:
use builtin 'indexed';
foreach my ($index, $val) (indexed @array) {
...
}
true and false return boolean true and false values. Perl is still perl, and doesn't have strict typing of booleans, but these values will be known to have been created as booleans. is_bool will tell you whether a value was known to have been created as a boolean.
These functions will, respectively: weaken a reference; strengthen a reference; and return whether a reference is weak. (A weak reference is not counted for garbage collection purposes. See perlref.) These can take the place of some similar routines in Scalar::Util.
These functions provide more data about references (or non-references, actually!) and can take the place of similar routines found in Scalar::Util.
ceil returns the smallest integer greater than or equal to its argument. floor returns the largest integer less than or equal to its argument. These can take the place of similar routines found in POSIX.
defer blocks (experimental)
This release adds support for defer blocks, which are blocks of code prefixed by the defer modifier. They provide a section of code which runs at a later time, during scope exit.
In brief, when a defer block is reached at runtime, its body is set aside to be run when the enclosing scope is exited. It is unlike a UNITCHECK (among other reasons) in that if the block containing the defer block is exited before the block is reached, it will not be run.
defer blocks can be used to take the place of "scope guard" objects where an object is passed a code block to be run by its destructor.
For more information, see "defer blocks" in perlsyn.
finally block (experimental)
The experimental try/catch syntax has been extended to support an optional third block introduced by the finally keyword.
try {
attempt();
print "Success\n";
}
catch ($e) {
print "Failure\n";
}
finally {
print "This happens regardless\n";
}
This provides code which runs at the end of the try/catch construct, even if aborted by an exception or control-flow keyword. They are similar to defer blocks.
For more information, see "Try Catch Exception Handling" in perlsyn.
Perl traditionally has allowed just four pairs of string/pattern delimiters: ( ){ }[ ]< >qr« »use utf8; q𝄃string𝄂
Even though subroutine signatures are now stable, use of the legacy arguments array (@_) with a subroutine that has a signature remains experimental, with its own warning category. Silencing the experimental::signatures warning category is not sufficient to dismiss this. The new warning is emitted with the category name experimental::args_array_with_signatures.
Any subroutine that has a signature and tries to make use of the defaults argument array or an element thereof (@_ or $_[INDEX]), either explicitly or implicitly (such as shift or pop with no argument) will provoke a warning at compile-time:
use v5.36;
sub f ($x, $y = 123) {
say "The first argument is $_[0]";
}
Use of @_ in array element with signatured subroutine is experimental at file.pl line 4.
The behaviour of code which attempts to do this is no longer specified, and may be subject to change in a future version.
@a = sort @empty; # unaffected @a = sort; # now a compile-time error @a = sort (); # also a compile-time error
A bare sort used to be a weird way to create an empty list; now it croaks at compile time. This change is intended to free up some of the syntax space for possible future enhancements to sort.
use VERSION (where VERSION is below v5.11) after use v5.11 is deprecated
When in the scope of use v5.11 or later, a use vX line where X is lower than v5.11 will now issue a warning:
Downgrading a use VERSION declaration to below v5.11 is deprecated
For example:
use v5.14; say "The say statement is permitted"; use v5.8; # This will print a warning print "We must use print\n";
This is because the Perl team plans to change the behavior in this case. Since Perl v5.12 (and parts of v5.11), strict is enabled unless it had previously been disabled. In other words:
no strict; use v5.12; # will not enable strict, because "no strict" preceded it $x = 1; # permitted, despite no "my" declaration
In the future, this behavior will be eliminated and use VERSION will always enable strict for versions v5.12 and later.
Code which wishes to mix versions in this manner should use lexical scoping with block syntax to ensure that the differently versioned regions remain lexically isolated.
{
use v5.14;
say "The say statement is permitted";
}
{
use v5.8; # No warning is emitted
print "We must use print\n";
}
Of course, this is probably not something you ever need to do! If the first block compiles, it means you're using perl v5.14.0 or later.
Configure run with the defaults will build an unthreaded Perl (which is slightly faster), but most operating systems ship a threaded Perl.
The same internal datatype (PVHV) is used for all of
The shared string table was originally added to improve performance for blessed hashes used as objects, because every object instance has the same keys, so it is an optimisation to share memory between them. It also makes sense for symbol tables, where derived classes will have the same keys (typically method names), and the OP trees built for method calls can also share memory. The shared string table behaves roughly like a cache for hash keys.
But for hashes actually used as associative arrays - mapping keys to values - typically the keys are not re-used in other hashes. For example, "seen" hashes are keyed by object IDs (or addresses), and logically these keys won't repeat in other hashes.
Storing these "used just once" keys in the shared string table increases CPU and RAM use for no gain. For such keys the shared string table behaves as a cache with a 0% hit rate. Storing all the keys there increases the total size of the shared string table, as well as increasing the number of times it is resized as it grows. Worse - in any environment that has "copy on write" memory for child process (such as a pre-forking server), the memory pages used for the shared string table rapidly need to be copied as the child process manipulates hashes. Hence if most of the shared string table is such that keys are used only in one place, there is no benefit from re-use within the perl interpreter, but a high cost due to more pages for the OS to copy.
The perl interpreter can now be Configured to disable shared hash keys for "large" hashes (that are neither objects nor symbol tables). To do so, add -Accflags='-DPERL_USE_UNSHARED_KEYS_IN_LARGE_HASHES' to your Configure options. "Large" is a heuristic -- currently the heuristic is that sharing is disabled when adding a key to a hash triggers allocation of more storage, and the hash has more than 42 keys.
This might cause slightly increased memory usage for programs that create (unblessed) data structures that contain multiple large hashes that share the same keys. But generally our testing suggests that for the specific cases described it is a win, and other code is unaffected.
For example, the following code is now executing ~30% faster:
$str = "A" x 64;
for (0..1_000_000) {
@svs = split //, $str
}
(You can read more about this one in [perl #19414].)
This document provides the process for administering an election or vote within the Perl Core Team.
We have attempted to update the documentation to reflect the changes listed in this document. If you find any we have missed, open an issue at https://github.com/Perl/perl5/issues.
Additionally, the following selected changes have been made:
As always, Patches Welcome!
:win32 IO pseudolayer is removed (this happened in 5.35.2).The following additions or changes have been made to diagnostic output, including warnings and fatal error messages. For the complete list of diagnostic messages, see perldiag.
(F) An attempt was made to jump out of the scope of a defer block by using a control-flow statement such as return, goto or a loop control. This is not permitted.
undef)
Attempting to perform a scalar assignment to undef, for example via undef = $foo;, previously triggered a fatal runtime error with the message "Modification of a read-only value attempted." It is more helpful to detect such attempted assignments prior to runtime, so they are now compile time errors, resulting in the message "Can't modify undef operator in scalar assignment".
The parser failed an internal consistency check while trying to parse a foreach loop.
A call is being made to a function in the builtin:: namespace, which is currently experimental.
The defer block modifier is experimental. If you want to use the feature, disable the warning with no warnings 'experimental::defer', but know that in doing so you are taking the risk that your code may break in a future Perl version.
This warning is emitted on a use VERSION statement that requests a version below v5.11 (when the effects of use strict would be disabled), after a previous declaration of one having a larger number (which would have enabled these effects)
This warning is emitted if you use for to iterate multiple values at a time. This syntax is currently experimental and its behaviour may change in future releases of Perl.
An expression that implicitly involves the @_ arguments array was found in a subroutine that uses a signature.
An expression involving the @_ arguments array was found in a subroutine that uses a signature.
Attempts to put wide characters into the program name ($0) now provoke this warning.
This warning used to not include the in %s.
Localized subroutine redefinitions no longer trigger this warning.
This makes it consistent with other checks of internal consistency when compiling a subroutine.
scalar category.
When sort is used in scalar context, it provokes a warning that doing this is not useful. This warning used to be in the void category. A new category for warnings about scalar context has now been added, called scalar.
Many diagnostics that have been removed from the perl core across many years have now also been removed from the documentation.
This may cause problems for some XS modules that unconditionally add -Werror=declaration-after-statement to their C compiler flags if compiling with gcc or clang. Earlier versions of Perl support long obsolete compilers that are strict in rejecting certain C99 features, particularly mixed declarations and code, and hence it makes sense for XS module authors to audit that their code does not violate this. However, doing this is now only possible on these earlier versions of Perl, hence these modules need to be changed to only add this flag for <$] < 5.035005>.
When using MAKEFLAGS=-j8, this significantly reduces the time required for:
sh ./makedepend MAKE=make cflags
#include <xlocale.h> is required to use the POSIX 1003 thread-safe locale functions or some related extensions. This prevents problems where a non-public xlocale.h is removed in a library update, or xlocale.h isn't intended for public use. (github #18936)Tests were added and changed to reflect the other additions and changes in this release.
These did not support C99 and hence can no longer be used to compile perl.
keys %ENV on VMS returns consistent results
On VMS entries in the %ENV hash are loaded from the OS environment on first access, hence the first iteration of %ENV requires the entire environment to be scanned to find all possible keys. This initialisation had always been done correctly for full iteration, but previously was not happening for %ENV in scalar context, meaning that scalar %ENV would return 0 if called before any other %ENV access, or would only return the count of keys accessed if there had been no iteration.
These bugs are now fixed - %ENV and keys %ENV in scalar context now return the correct result - the count of all keys in the environment.
UWIN is a UNIX compatibility layer for Windows. It was last released in 2012 and has been superseded by Cygwin these days.
DJGPP is a port of the GNU toolchain to 32-bit x86 systems running DOS. The last known attempt to build Perl on it was on 5.20, which only got as far as building miniperl.
Support code for Novell NetWare has been removed. NetWare was a server operating system by Novell. The port was last updated in July 2002, and the platform itself in May 2009.
Unrelated changes accidentally broke the build for the NetWare port in September 2009, and in 12 years no-one has reported this.
This update enables us to build EBCDIC static/dynamic and 31-bit/64-bit addressing mode Perl. The number of tests that pass is consistent with the baseline before these updates.
These changes also provide the base support to be able to provide ASCII static/dynamic and 31-bit/64-bit addressing mode Perl.
The z/OS (previously called OS/390) README was updated to describe ASCII and EBCDIC builds.
PERL_IMPLICIT_CONTEXT will remain defined for compatibility with XS modules.
G_ARRAY, indicating list context, has now been renamed to a more accurate G_LIST. A compatibilty macro G_ARRAY has been added to allow existing code to work unaffected. New code should be written using the new constant instead. This is supported by Devel::PPPort version 3.63.PERL_GCC_VERSION_GE, PERL_GCC_VERSION_GT, PERL_GCC_VERSION_LE and PERL_GCC_VERSION_LT.
Inline functions have been added to embed.h to determine the position of the least significant 1 bit in a word: lsbit_pos32 and lsbit_pos64.
Perl_ptr_table_clear has been deleted. This has been marked as deprecated since v5.14.0 (released in 2011), and is not used by any code on CPAN.xs_handshake()'s two failure modes now provide distinct messages.struct xpvhv_aux) is now allocated as part of the hash body, instead of as part of the block of memory allocated for the main hash array.pack behavior of U has changed for EBCDIC.sv_numeq and sv_streq have been added, along with ..._flags-suffixed variants. These expose a simple and consistent API to perform numerical or string comparison which is aware of operator overloading.SVf_POK. The string form is still cached internally, and still re-read directly by the macros SvPV(sv) etc (inline, without calling a C function). XS code that already calls the APIs to get values will not be affected by this change. XS code that accesses flags directly instead of using API calls to express its intent might break, but such code likely is already buggy if passed some other values, such as floating point values or objects with string overloading.This small change permits code (such as JSON serializers) to reliably determine between
my $answer = 42; print "The answer is $answer\n";
my $answer = "42";
print "That doesn't look right\n"
unless $answer == 6 * 9;
For the first case (originally written as an integer), we now have:
use Devel::Peek; my $answer = 42; Dump ($answer); my $void = "$answer"; print STDERR "\n"; Dump($answer) SV = IV(0x562538925778) at 0x562538925788 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 42 SV = PVIV(0x5625389263c0) at 0x562538925788 REFCNT = 1 FLAGS = (IOK,pIOK,pPOK) IV = 42 PV = 0x562538919b50 "42"\0 CUR = 2 LEN = 10
For the second (originally written as a string), we now have:
use Devel::Peek; my $answer = "42"; Dump ($answer); my $void = $answer == 6 * 9; print STDERR "\n"; Dump($answer)' SV = PV(0x5586ffe9bfb0) at 0x5586ffec0788 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x5586ffee7fd0 "42"\0 CUR = 2 LEN = 10 COW_REFCNT = 1 SV = PVIV(0x5586ffec13c0) at 0x5586ffec0788 REFCNT = 1 FLAGS = (IOK,POK,IsCOW,pIOK,pPOK) IV = 42 PV = 0x5586ffee7fd0 "42"\0 CUR = 2 LEN = 10 COW_REFCNT = 1
(One can't rely on the presence or absence of the flag SVf_IsCOW to determine the history of operations on a scalar.)
Previously both cases would be indistinguishable, with all 4 flags set:
SV = PVIV(0x55d4d62edaf0) at 0x55d4d62f0930 REFCNT = 1 FLAGS = (IOK,POK,pIOK,pPOK) IV = 42 PV = 0x55d4d62e1740 "42"\0 CUR = 2 LEN = 10
(and possibly SVf_IsCOW, but not always)
This now means that if XS code really needs to determine which form a value was first written as, it should implement logic roughly
if (flags & SVf_IOK|SVf_NOK) && !(flags & SVf_POK)
serialize as number
else if (flags & SVf_POK)
serialize as string
else
the existing guesswork ...
Note that this doesn't cover "dualvars" - scalars that report different values when asked for their string form or number form (such as $!). Most serialization formats cannot represent such duplicity.
The existing guesswork remains because as well as dualvars, values might be undef, references, overloaded references, typeglobs and other things that Perl itself can represent but do not map one-to-one into external formats, so need some amount of approximation or encapsulation.
sv_dump (and Devel::Peek’s Dump function) now escapes high-bit octets in the PV as hex rather than octal. Since most folks understand hex more readily than octal, this should make these dumps a bit more legible. This does not affect any other diagnostic interfaces like pv_display.$! when called on a closed handle.mktables debugging facility would cause perl to croak since v5.31.10; this problem has now been fixed.makedepend logic is now compatible with BSD make (fixes GH #19046).untie on a tied hash that is partway through iteration now frees the iteration state immediately.
Iterating a tied hash causes perl to store a copy of the current hash key to track the iteration state, with this stored copy passed as the second parameter to NEXTKEY. This internal state is freed immediately when tie hash iteration completes, or if the hash is destroyed, but due to an implementation oversight, it was not freed if the hash was untied. In that case, the internal copy of the key would persist until the earliest of
tie was called again on the same hashkeys, values or each)
This inconsistency is now fixed - the internal state is now freed immediately by untie.
As the precise timing of this behaviour can be observed with pure Perl code (the timing of DESTROY on objects returned from FIRSTKEY and NEXTKEY) it's just possible that some code is sensitive to it.
Internals::getcwd() function added for bootstrapping miniperl in perl 5.30.0 is now only available in miniperl. [github #19122]use statement could cause a memory write to a freed dbstate op. [GH #19198]Raun "Spider" Boardman (SPIDB on CPAN), author of at least 66 commits to the Perl 5 core distribution between 1996 and 2002, passed away May 24, 2021 from complications of COVID. He will be missed.
David H. Adler (DHA) passed away on November 16, 2021. In 1997, David co-founded NY.pm, the first Perl user group, and in 1998 co-founded Perl Mongers to help establish other user groups across the globe. He was a frequent attendee at Perl conferences in both North America and Europe and well known for his role in organizing Bad Movie Night celebrations at those conferences. He also contributed to the work of the Perl Foundation, including administering the White Camel awards for community service. He will be missed.
Perl 5.36.0 represents approximately a year of development since Perl 5.34.0 and contains approximately 250,000 lines of changes across 2,000 files from 82 authors.
Excluding auto-generated files, documentation and release tools, there were approximately 190,000 lines of changes to 1,300 .pm, .t, .c and .h files.
Perl continues to flourish into its fourth decade thanks to a vibrant community of users and developers. The following people are known to have contributed the improvements that became Perl 5.36.0:
Alyssa Ross, Andrew Fresh, Aristotle Pagaltzis, Asher Mancinelli, Atsushi Sugawara, Ben Cornett, Bernd, Biswapriyo Nath, Brad Barden, Bram, Branislav Zahradník, brian d foy, Chad Granum, Chris 'BinGOs' Williams, Christian Walde (Mithaldu), Christopher Yeleighton, Craig A. Berry, cuishuang, Curtis Poe, Dagfinn Ilmari Mannsåker, Dan Book, Daniel Laügt, Dan Jacobson, Dan Kogai, Dave Cross, Dave Lambley, David Cantrell, David Golden, David Marshall, David Mitchell, E. Choroba, Eugen Konkov, Felipe Gasper, François Perrad, Graham Knop, H.Merijn Brand, Hugo van der Sanden, Ilya Sashcheka, Ivan Panchenko, Jakub Wilk, James E Keenan, James Raspass, Karen Etheridge, Karl Williamson, Leam Hall, Leon Timmermans, Magnus Woldrich, Matthew Horsfall, Max Maischein, Michael G Schwern, Michiel Beijen, Mike Fulton, Neil Bowers, Nicholas Clark, Nicolas R, Niyas Sait, Olaf Alders, Paul Evans, Paul Marquess, Petar-Kaleychev, Pete Houston, Renee Baecker, Ricardo Signes, Richard Leach, Robert Rothenberg, Sawyer X, Scott Baker, Sergey Poznyakoff, Sergey Zhmylove, Sisyphus, Slaven Rezic, Steve Hay, Sven Kirmess, TAKAI Kousuke, Thibault Duponchelle, Todd Rinaldo, Tomasz Konojacki, Tomoyuki Sadahiro, Tony Cook, Unicode Consortium, Yves Orton, Михаил Козачков.
The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.
For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.
If you find what you think is a bug, you might check the perl bug database at https://github.com/Perl/perl5/issues. There may also be information at http://www.perl.org/, the Perl Home Page.
If you believe you have an unreported bug, please open an issue at https://github.com/Perl/perl5/issues. Be sure to trim your bug down to a tiny but sufficient test case.
If the bug you are reporting has security implications which make it inappropriate to send to a public issue tracker, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec for details of how to report the issue.
If you wish to thank the Perl 5 Porters for the work we had done in Perl 5, you can do so by running the perlthanks program:
perlthanks
This will send an email to the Perl 5 Porters list with your show of thanks.
The Changes file for an explanation of how to view exhaustive details on what changed.
The INSTALL file for how to build Perl.
The README file for general stuff.
The Artistic and Copying files for copyright information.