Optimizing the number of font glyphs for web use

/en/linux/how-to/ fontforgeglyphpyftsubsetwofffont
12/March/2020
authorRentaro
Rentaro
Select the required glyphs 1590x814 PTSansNarrowRegular.png
fontforge glyphs

Required packages apt install fontforge и pip install fonttools.

Let’s decide on the necessary glyphs: we need Latin, Cyrillic, special characters, including some Greek symbols (ΣΩαβγμ), Greek analogues that are similar in style but have different codes (∆∑µ), and a few more that can be used somewhere in the text or as elements to replace graphics (♪♫♬).

Why are there multiple variants of similar glyphs with different codes?

Because the µ (mu U+00b5) symbol and the Greek letter μ (mu U+03bc) in extended fonts can have their own glyph with different shapes, or, conversely, a common glyph, where similar characters in shape refer to the same glyph. Or, for example, the Greek Σ (Sigma U+03a3) is not the same as the symbol ∑ (summation U+2211).
But perhaps these symbols look the same on your screen.

Example with prepared character sets in unicode

nano make_menu.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

#!/bin/bash

OPTS=`getopt -o I:O:S:L: --long in:,out:,subset:,layout: -n 'parse-options' -- "$@"` ; EC="$?"
eval set -- "$OPTS"
while true; do
  case "$1" in
    -I | --in )                   in=$2;                      shift 2 ;;
    -O | --out)                   out=$2;                     shift 2 ;;
    -S | --subset)                subset=$2;                  shift 2 ;;
    -L | --layout)                layout=$2;                  shift 2 ;;
    -- ) shift; break ;;
    *  ) break ;;
  esac
done

printf 'Read:   %s\n' "${in}"
printf 'Write:  %s\n' "${out}"

base="U+20-7E,U+A1-B7,U+B9-BE,U+C0,U+C1,U+C8,U+C9,\
U+CC,U+CD,U+D2,U+D3,U+D7,U+D9,U+DA,U+E0,U+E1,U+E8,U+E9,U+EB-ED,\
U+EF,U+F2,U+F3,U+F7-FA,U+131,U+2BB,U+2BC,U+2C6,U+2DA,U+2DC,U+400,U+401,\
U+403-408,U+410-451,U+453-458,U+45C-45F,U+490,U+491,U+4B0,U+4B1,\
U+2011,U+2018-201F,U+2022-2026,U+2030-203A,U+2041,U+2043,U+2044,\
U+204B,U+2052,U+2053,U+2074,U+20AC,U+2116,U+2122,U+2191,U+2193,U+2212,U+2215"

base_ext="U+20-7E,U+A1-B7,U+B9-BE,U+C0,U+C1,U+C8,U+C9,\
U+CC,U+CD,U+D2,U+D3,U+D7,U+D9,U+DA,U+E0,U+E1,U+E8,U+E9,\
U+EB-ED,U+EF,U+F2,U+F3,U+F7-FA,U+131,U+192,U+2BB,U+2BC,U+2C6,U+2C7,\
U+2D9,U+2DA,U+2DC,U+394,U+3A9,U+3C0,U+400,U+401,U+403-408,U+410-451,\
U+453-458,U+45C-45F,U+462,U+463,U+490,U+491,U+4B0,U+4B1,U+2011,\
U+2018-201F,U+2022-2026,U+2030-203A,U+2041,U+2043,U+2044,U+204B,\
U+2052,U+2053,U+2074,U+2081-2084,U+20AC,U+20B4,U+20BD,U+20BF,\
U+2116,U+2122,U+212E,U+2191,U+2193,U+2211,U+2212,U+2215,U+221A,U+221E,\
U+222B,U+2248,U+2260,U+2264,U+2265"

menu="U+20-7E,U+A3,U+A5,U+A7,U+A9,U+AB,U+AD,U+AE,U+B0,U+B4,U+B5,U+BB,\
U+C0,U+C1,U+C8,U+C9,U+CC,U+CD,U+D2,U+D3,U+D7,U+D9,U+DA,U+E0,U+E1,\
U+E8,U+E9,U+EB-ED,U+EF,U+F2,U+F3,U+F7-FA,U+131,U+192,\
U+2BB,U+2BC,U+2C6,U+2C7,U+2D9,U+2DA,\U+2DC,U+400,U+401,U+403-408,U+410-451,\
U+453-458,U+45C-45F,U+462,U+463,U+490,U+491,U+4B0,U+4B1,U+2011,U+2018-201F,\
U+2022-2026,U+2030-2037,U+2039,U+203A,U+2041,U+2043,U+2052,U+2053,U+20AC,U+20B4,\
U+20BD,U+20BF,U+2116,U+2122,U+2191,U+2193,U+221A,U+221E,U+222B,U+2248,U+2260,U+2264,U+2265"

menu_textonly="U+20-7E,U+A9,U+AB,U+AD,U+AE,U+B0,U+B4,U+BB,U+C0,U+C1,U+C8,U+C9,U+CC,U+CD,\
U+D2,U+D3,U+D9,U+DA,U+E0,U+E1,U+E8,U+E9,U+EB-ED,U+EF,U+F2,U+F3,U+F9,U+FA,U+131,U+400,U+401,U+403-408,U+410-451,\
U+453-458,U+45C-45E,U+490,U+491,U+2024-2026,U+2039,U+203A,U+20AC,U+20B4,U+20BD,U+20BF"

if [[ "$subset" == "base" ]]; then
    subset_list=$base
fi
if [[ "$subset" == "base_ext" ]]; then
    subset_list=$base_ext
fi
if [[ "$subset" == "menu" ]]; then
    subset_list=$menu
fi
if [[ "$subset" == "menu_textonly" ]]; then
    subset_list=$menu_textonly
fi
layout_add="kern,liga,clig,calt,ccmp,locl,mark,mkmk,onum,pnum,smcp,c2sc,lnum,tnum,subs,sups,cswh,dlig,ss01,ss03,zero"
layout_rem="dnom,numr"
if [[ "$layout" == "web" ]]; then
    layout_add="kern,liga,clig,calt,ccmp,locl,mark,mkmk,lnum,tnum,cswh,dlig,ss01,ss03,zero"
    layout_rem="dnom,numr,frac,onum,pnum,smcp,c2sc,subs,sups"
fi

/usr/bin/pyftsubset "${in}" --output-file="${out}" \
    --unicodes="${subset_list}" \
    --layout-features+="${layout_add}" \
    --layout-features-="${layout_rem}"

#     --flavor=woff2 \
#    --no-hinting \
#    --desubroutinize \

nano ff-conv.sh

1
2
3
4
5
6
7
8

#!/usr/bin/fontforge
Open($1)
Generate($1:r + ".otf")
Generate($1:r + ".svg")
Generate($1:r + ".woff")
Generate($1:r + ".woff2")
Generate($1:r + ".eot")
Generate($1:r + ".ttf")

This site uses the Latin alphabet, but not with all symbols, such as diacritics, and also Cyrillic and some special characters.

Symbolic representation of Unicode characters

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# base_ext
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¹
º»¼½¾ÀÁÈÉÌÍÒÓ×ÙÚàáèéëìíïòó÷øùúıƒʻʼˆˇ˙˚˜ΔΩπЀЁЃЄЅІЇЈАБВГДЕЖЗИ
ЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяѐёѓє
ѕіїјќѝўџѢѣҐґҰұ‑‘’‚‛“”„‟•‣․‥…‰‱′″‴‵‶‷‸‹›⁁⁃⁄⁋⁒⁓⁴₁₂₃₄€₴₽₿№™℮
↑↓∑−∕√∞∫≈≠≤≥

# base_unicode
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¹
º»¼½¾ÀÁÈÉÌÍÒÓ×ÙÚàáèéëìíïòó÷øùúıʻʼˆ˚˜ЀЁЃЄЅІЇЈАБВГДЕЖЗИЙКЛМНО
ПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяѐёѓєѕіїјќѝ
ўџҐґҰұ‑‘’‚‛“”„‟•‣․‥…‰‱′″‴‵‶‷‸‹›⁁⁃⁄⁋⁒⁓⁴€№™↑↓−∕

# menu
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~£¥§©«­®°´µ»ÀÁÈÉÌÍÒÓ×ÙÚàá
èéëìíïòó÷øùúıƒʻʼˆˇ˙˚˜ЀЁЃЄЅІЇЈАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭ
ЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяѐёѓєѕіїјќѝўџѢѣҐґҰұ‑‘’‚‛“”
„‟•‣․‥…‰‱′″‴‵‶‷‹›⁁⁃⁒⁓€₴₽₿№™↑↓√∞∫≈≠≤≥

# menu_textonly
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~©«­®°´»ÀÁÈÉÌÍÒÓÙÚàáèéëìí
ïòóùúıЀЁЃЄЅІЇЈАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклм
нопрстуфхцчшщъыьэюяѐёѓєѕіїјќѝўҐґ․‥…‹›€₴₽₿

Generating fonts

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

./make_menu.sh --subset "base_ext" \
    --in PTSansNarrowRegular_original.ttf \
    --out original/PTSansNarrowRegular_subset.ttf
./ff-conv.sh original/PTSansNarrowRegular_subset.ttf

./make_menu.sh --subset "base_ext" \
    --in FiraSansExtraCondensedLight_original.ttf \
    --out original/FiraSansExtraCondensedLight_subset.ttf
./ff-conv.sh original/FiraSansExtraCondensedLight_subset.ttf

./make_menu.sh --subset "base_ext" \
    --in LiberationMonoRegular_original.ttf \
    --out original/LiberationMonoRegular_subset.ttf
./ff-conv.sh original/LiberationMonoRegular_subset.ttf

./make_menu.sh --subset "menu_textonly" \
    --in ShantellSansNormalRegular_original.otf \
    --out original/ShantellSansNormalRegular_subset.otf
./ff-conv.sh original/ShantellSansNormalRegular_subset.otf

Original font PTSansNarrowRegular_original.ttf 230K

1
2
3
4
5
6
7

  87K PTSansNarrowRegular_subset.afm
  44K PTSansNarrowRegular_subset.eot
  71K PTSansNarrowRegular_subset.otf
 129K PTSansNarrowRegular_subset.svg
 101K PTSansNarrowRegular_subset.ttf
  52K PTSansNarrowRegular_subset.woff
  40K PTSansNarrowRegular_subset.woff2

Some fonts may use a single glyph for certain character combinations,
for Latin, these are often combinations of fi fl ff ft.
Often, web font generation services remove these glyphs.
If you have this problem, you can solve it by simply adding a CSS rule:
letter-spacing:-0.1pt;

fl-fi-ft
fl-fi-ft 1590x814
font-glyph-fl-fi-ft.png
fl-fi-ft error
fl-fi-ft error 400x110
font-fl-fi-ft-errore.png

Example with manual code selection

Open the .ttf file in Fontforge and begin selecting and copying the desired glyph numeric codes into the file.

Select the required glyphs 1290x753 fontforge.png
fontforge glyphs 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# latin range
awk 'BEGIN{for(i=32;i<127;i++)printf "%c",i}' > glyph-symbols

# cyrillic range
awk 'BEGIN{for(i=1040;i<1106;i++)printf "%c",i}' >> glyph-symbols

# special range
awk 'BEGIN{for(i=8216;i<8223;i++)printf "%c",i}' >> glyph-symbols

# special
A="160 161 166 167 169 171 173 174 176 177 178 179 180 181 183 185 187 \
188 189 190 198 215 216 230 247 248 402 729 730 732 733 916 931 937 \
945 946 947 948 949 955 956 957 960 966 968 969 1025 1027 1028 1030 \
1031 1037 1107 1108 1110 1111 1122 1123 1168 1169 8240 8226 8230 8240 \
8242 8243 8249 8250 8356 8364 8372 8381 8470 8482 8486 8494 8592 8593 \
8594 8595 8596 8597 8710 8721 8722 8725 8730 8734 8747 8776 8800 8804 \
8805 9834 9835 9836" ; awk -v var="${A[*]}" 'BEGIN{split(var,list," "); \
    for (i=1;i<=length(list);i++) printf "%c", list[i]}' \
        >> glyph-symbols

pyftsubset PTSansNarrow-Regular.ttf --output-file=PTSans-sub.ttf --text-file=glyph-symbols

Universal CSS style for font and fallback with size adjustments

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

@font-face {
        font-family: "ShantellSansNormalRegular";
        src:  url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.eot");                                               /* IE9 Compat Modes */
        src:  url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.eot?#iefix") format("embedded-opentype"),            /* IE6-IE8 */
              url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.woff2") format("woff2"),                             /* Modern Browsers */
              url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.woff") format("woff"),                               /* Modern Browsers */
              url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.ttf") format("truetype"),                            /* Safari, Android, iOS */
              url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.svg#ShantellSansNormal-Regular") format("svg");      /* Legacy iOS */
        src:  url("/assets/css/fonts/original/ShantellSansNormalRegular_subset.otf") format(opentype) tech(incremental-range);      /* Open Type Font */
        font-weight:normal;
        font-style:normal;
        font-display:block;
        unicode-range:U+20-7E,U+A9,U+AB,U+AD,U+AE,U+B0,U+B4,U+BB,U+C0,U+C1,U+C8,U+C9,U+CC,U+CD,U+D2,U+D3,U+D9,U+DA,U+E0,U+E1,U+E8,U+E9,U+EB-ED,U+EF,U+F2,U+F3,U+F9,U+FA,U+131,U+400,U+401,U+403-408,U+410-451,U+453-458,U+45C-45E,U+490,U+491,U+2024-2026,U+2039,U+203A,U+20AC,U+20B4,U+20BD,U+20BF;}

@font-face {
        font-family: "fallbackShantellComic";
        src: local("Comic Sans MS");
        size-adjust: 98%;
        ascent-override: 107%;
        descent-override: 25%;
        line-gap-override: 5%;
}
@font-face {
        font-family: "fallbackShantellArial";
        src: local("Arial");
        size-adjust: 105%;
        ascent-override: 100%;
        descent-override: 29%;
        line-gap-override: 5%;
}

p {font-family:"ShantellSansNormalRegular", "fallbackShantellComic", "fallbackShantellArial", "Comic Sans MS", sans-serif;}

And a little more

Count the number of glyphs in the original file and the one we made ourselves.

1
2
3
4
5
6
7
8
9
10
11

fontforge -lang=ff -c 'Open($1); SelectWorthOutputting(); Print($selection)' \
    "PTSansNarrowRegular_original.ttf" 2>/dev/null | \
    tr -d '][' | tr , '\n' | grep -c 1

723

fontforge -lang=ff -c 'Open($1); SelectWorthOutputting(); Print($selection)' \
    "PTSansNarrowRegular_subset.ttf" 2>/dev/null | \
    tr -d '][' | tr , '\n' | grep -c 1

292

For .svg, this can be done, for example, like this:

cat PTSansNarrow.svg | grep "glyph-name" | wc -l
723
cat PTSansNarrow_subset.svg | grep "glyph-name" | wc -l
292

It should be noted here that the resulting fonts may contain fewer glyphs than the characters for selection, because a specific font may not have glyphs for some characters. The script contains an extended set of characters (codes) for selection from different fonts, but the main thing is that it does not contain unnecessary characters. However, it would still be more correct to open a specific font in the editor and select the required glyphs from the available ones.

If the font is intended to be used only for text, which sometimes does not even require additional characters to alphanumeric other than the period and comma, then the number of glyphs can be reduced even further.

1
2
3
4
5
6
7
8
9

# Latin
awk 'BEGIN{for(i=32;i<127;i++)printf "%c",i}' > short-glyphs
# Cyrillic
awk 'BEGIN{for(i=1040;i<1106;i++)printf "%c",i}' >> short-glyphs
A="1025 1028 1030 1031 1108 1110 1111 1168 1169" ; awk -v var="${A[*]}" 'BEGIN{split(var,list," "); for (i=1;i<=length(list);i++) printf "%c", list[i]}'  >> short-glyphs

pyftsubset Caveat-Regular.ttf \
    --output-file=Caveat-Regular-short.ttf \
    --text-file=short-glyphs