NAME
Unicode::Confuse - Identify and replace Unicode confusables
SYNOPSIS
use utf8;
use Unicode::Confuse ':all';
if (confusable ('ρ')) {
my $canonical = canonical ('ρ');
print "'ρ' is confusable with $canonical.\n";
my @similar = similar ($canonical);
print "$canonical is also confusable with @similar.\n";
}
produces output
'ρ' is confusable with p.
p is also confusable with p ρ ϱ р ⍴ ⲣ ｐ 𝐩 𝑝 𝒑 𝓅 𝓹 𝔭 𝕡 𝖕 𝗉 𝗽 𝘱 𝙥 𝚙 𝛒 𝛠 𝜌 𝜚 𝝆 𝝔 𝞀 𝞎 𝞺 𝟈.(This example is included as synopsis.pl in the distribution.)
VERSION
This documents version 0.05 of Unicode-Confuse corresponding to git commit 4a97ea6b65f148a559b21ebde80b189257d16b1c released on Thu Apr 29 11:49:01 2021 +0900.
This Perl module incorporates Unicode Security Mechanisms for UTS #39 version 13.0.0 dated 2020-02-13, 01:38:49 GMT, copyright © 2020 Unicode®, Inc.. For terms of use, see http://www.unicode.org/terms_of_use.html.
DESCRIPTION
This module offers functions for dealing with Unicode "confusables", characters which look similar to one another but are represented by different Unicode code points.
FUNCTIONS
canonical
my $canonical = canonical ($c);If $c is a confusable, give the canonical form of $c. If $c is already the canonical form of itself, return $c. If $c is not a confusable, this returns the undefined value. "Canonical" here just means the character which is used as a representative of the group of confusables in the "Unicode data files".
confusable
if (confusable ($c)) {
# do something.
}This returns a true or false value depending on whether $c is a confusable. This matches $c against a large regex in Unicode::Confuse::Regex.
similar
my @similar = similar ('p');Return a list of confusables which are similar to the given input. If the input is not a confusable, an empty list is returned.
The first character in @similar is the canonical form, and the remaining characters are the other confusables associated with that canonical form. These remaining characters, if more than one, are sorted by code point.
Example: obfuscate text
This example obfuscates strings by substituting confusable letters with substitutes picked at random from the confusable data for the letters.
use utf8;
use Unicode::Confuse ':all';
sub obfuscate
{
for (@_) {
my @letters = split '', $_;
my $out = '';
my $ok;
for my $letter (@letters) {
my @similar = similar ($letter);
if (@similar) {
$ok = 1;
my $n = scalar (@similar);
my $r = int (rand ($n));
$out .= $similar[$r];
}
else {
$out .= $letter;
}
}
if (! $ok) {
print "No confusables in '$_'.\n";
}
else {
print "$_ -> $out\n";
}
}
}
obfuscate ('paypal', '月火水木金土日');produces output
paypal -> 𝘱𝒂𝐲𝓅𝖺𝜤
月火水木金土日 -> 月火水木金⼠⽇(This example is included as obfuscate.pl in the distribution.)
DEPENDENCIES
- File::Slurper
-
This is used by the parsing module Unicode::Confuse::Parse.
- JSON::Parse
-
This is used to parse the JSON-formatted file of confusables distributed with the module.
BUGS
- Unicode specifications
-
This does not even attempt to replicate the Unicode requirements for software for handling confusables. In other words, this Perl module makes no claim whatsoever to be "An implementation claiming conformance to this specification" as described in the text of the "Unicode Consortium specification".
- Data quality
-
The data in the Unicode confusables file is of mixed quality, with nearly identical or indistinguishable characters muddled together with things which are clearly quite different from one another.
SEE ALSO
In this distribution
The script make-confusables.pl, available only in the github repository, makes the data files for this Perl distribution.
Unicode::Confuse::Parse is used to parse the data file (confusables.txt). It is used by make-confusables.pl.
Unicode::Confuse::Regex is generated by make-confusables.pl. It matches all confusables.
The unexported variable $Unicode::Confuse::data contains the complete confusable data. $data->{confusables} contains a map from confusables to the canonical format, and $data->{reverse} contains a map from the canonical form to an array containing the corresponding set of confusables, which may have only one member.
Unicode Consortium information
- Unicode Consortium specification
-
See http://www.unicode.org/reports/tr39 for the Unicode Consortium specification.
- Unicode data files
-
The following links point to the latest data files:
- Unicode Utilities: Confusables
-
The utility says
With this demo, you can supply an Input string and see the combinations that are confusable with it, using data collected by the Unicode consortium.
AUTHOR
Ben Bullock, <bkb@cpan.org>
COPYRIGHT & LICENCE
This package and associated files are copyright (C) 2021 Ben Bullock.
You can use, copy, modify and redistribute this package and associated files under the Perl Artistic Licence or the GNU General Public Licence.