如何构建一个简单的分词器

我想知道如何构建一个非常简单的标记器。给定字典 d（在本例中为列表）和句子 s，我想返回该句子的所有可能标记（=单词）。这是我尝试过的：

l = ["the","snow","ball","snowball","is","cold"]
sentence = "thesnowballisverycold"

def subs(string, ret=['']):
    if len(string) == 0:
        return ret
    head, tail = string[0], string[1:]
    ret = ret + list(map(lambda x: x+head, ret))
    return subs(tail, ret)
    
print((list(set(subs(sentence))&set(l))))

但这会返回：

["snow","ball","cold","is","snowball","the"]

我可以比较子字符串，但必须有更好的方法来做到这一点，对吗？我想要的是：

["the","snowball","is","cold"]

回答

import re
l = ["the","snow","ball","snowball","is","cold"]
pattern = "|".join(sorted(l, key=len, reverse=True))
sentence = "thesnowballisverycold"
print( re.findall(pattern, sentence) )
# => ['the', 'snowball', 'is', 'cold']