ActiveSupport::Multibyte::Unicode

Methods

compose
decompose
tidy_bytes

Constants

UNICODE_VERSION	=	RbConfig::CONFIG["UNICODE_VERSION"]
	The `Unicode` version that is supported by the implementation

Instance Public methods

compose(codepoints)

Compose decomposed characters to the composed form.

📝 Source code

# File activesupport/lib/active_support/multibyte/unicode.rb, line 21
      def compose(codepoints)
        codepoints.pack("U*").unicode_normalize(:nfc).codepoints
      end

🔎 See on GitHub

decompose(type, codepoints)

Decompose composed characters to the decomposed form.

📝 Source code

# File activesupport/lib/active_support/multibyte/unicode.rb, line 12
      def decompose(type, codepoints)
        if type == :compatibility
          codepoints.pack("U*").unicode_normalize(:nfkd).codepoints
        else
          codepoints.pack("U*").unicode_normalize(:nfd).codepoints
        end
      end

🔎 See on GitHub

tidy_bytes(string, force = false)

Replaces all ISO-8859-1 or CP1252 characters by their UTF-8 equivalent resulting in a valid UTF-8 string.

Passing true will forcibly tidy all bytes, assuming that the string’s encoding is entirely CP1252 or ISO-8859-1.

📝 Source code

# File activesupport/lib/active_support/multibyte/unicode.rb, line 30
      def tidy_bytes(string, force = false)
        return string if string.empty? || string.ascii_only?
        return recode_windows1252_chars(string) if force
        string.scrub { |bad| recode_windows1252_chars(bad) }
      end

🔎 See on GitHub