LIME for Text

 

여기서 model은 layer에 text vectorization layer가 들어있어

input 으로 raw 데이터인 text를 그대로 집어 넣어도 결과가 나온다.

 

text를 따로 vector변환하여 모델에 넣는 것을 구성하였다면, 

파이프라인을 만들어 전처리과정을 태우는 모델을 넣던지

아니면 input data를 직접 벡터화하여 넣는 방안도 있다.

두번째 안은 직접벡터화하면 plot에 벡터화된 숫자가 나올 것이므로 추천하지 않는다.

 

In [ ]:

!pip3 install lime

 

In [ ]:

!gsutil cp -r gs://exaple/model/20200729/KOR/text_classification_train_20200729_1596178422_KOR/keras_export/keyed_model/* ./

In [1]:

from lime.lime_text import LimeTextExplainer

import os
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers.experimental.preprocessing import TextVectorization

import matplotlib
import matplotlib.pyplot as plt
from matplotlib import font_manager

In [2]:

class_names=['부정','긍정']

In [3]:

explainer = LimeTextExplainer(class_names=class_names)

In [4]:

KEYED_EXPORT_PATH = './'

In [5]:

loaded_model = tf.keras.models.load_model(KEYED_EXPORT_PATH, custom_objects={"TextVectorization":TextVectorization})

In [6]:

sample_data=['별 흥미 를 못 끌 더니 시청 률 도 결국 참패','The movie is very nice. Actor is so handsome']

In [7]:

loaded_model.predict(sample_data)[0]

Out[7]:

array([0.9988205 , 0.00117953], dtype=float32)

In [8]:

exp1 = explainer.explain_instance(sample_data[0],loaded_model.predict, num_features=48)

 

/home/jupyter/.local/lib/python3.5/site-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]

In [9]:

font_dirs = ['./']
font_files = font_manager.findSystemFonts(fontpaths=font_dirs)
font_list = font_manager.createFontList(font_files)
font_manager.fontManager.ttflist.extend(font_list)
font_list

Out[9]:

[<Font 'NanumBarunGothic' (NanumBarunGothic.ttf) normal normal 400 normal>]

In [10]:

plt.rcParams["font.family"] = "NanumBarunGothic"
fig = exp1.as_pyplot_figure()
exp1.show_in_notebook(text=False)

 

 

 

In [11]:

exp2 = explainer.explain_instance(sample_data[1],loaded_model.predict, num_features=48)

 

/home/jupyter/.local/lib/python3.5/site-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]

In [12]:

fig = exp2.as_pyplot_figure()
exp2.show_in_notebook(text=False)