Nelf Name Generator

I have taken a look into LSTM-RNN for the works of ICT&Cambricon. I found the LSTM language model is kind of attractive to me.

I have built a name generator for Nelf female. Export all Nelf female NPC names from a private server database(Arkcore NG, 4.3.2), and delete last names and titles manually, just save all first names. There are 498 first names at total(maybe have few duplicates).

Run a python script to generate a random sequence of 200,000 names as training data. Build a mini LSTM network with 2 layers, each has 1000 neurons, and a 50% drop rate after each layer.


name: "lstm_language_model"
layer {
  name: "data"
  type: "HDF5Data"
  top: "cont_sentence"
  top: "input_sentence"
  top: "target_sentence"
  include { phase: TRAIN }
  hdf5_data_param {
    source: "hdf5_list.txt"
    batch_size: 200
  }
}
layer {
  name: "data"
  type: "HDF5Data"
  top: "cont_sentence"
  top: "input_sentence"
  top: "target_sentence"
  include {
    phase: TEST
  }
  hdf5_data_param {
    source: "hdf5_list.txt"
    batch_size: 1
  }
}
layer {
  name: "lstm1"
  type: "LSTM"
  bottom: "input_sentence"
  bottom: "cont_sentence"
  top: "lstm1"
  recurrent_param {
    num_output: 1000
    weight_filler {
      type: "uniform"
      min: -0.08
      max: 0.08
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "lstm1-drop"
  type: "Dropout"
  bottom: "lstm1"
  top: "lstm1"
  dropout_param { dropout_ratio: 0.5 }
  include { stage: "lstm-drop" }
}
layer {
  name: "lstm2"
  type: "LSTM"
  bottom: "lstm1"
  bottom: "cont_sentence"
  top: "lstm2"
  recurrent_param {
    num_output: 1000
    weight_filler {
      type: "uniform"
      min: -0.08
      max: 0.08
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "lstm2-drop"
  type: "Dropout"
  bottom: "lstm2"
  top: "lstm2"
  dropout_param { dropout_ratio: 0.5 }
  include { stage: "lstm-drop" }
}
layer {
  name: "predict"
  type: "InnerProduct"
  bottom: "lstm2"
  top: "predict"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 29
    weight_filler {
      type: "uniform"
      min: -0.08
      max: 0.08
    }
    bias_filler {
      type: "constant"
      value: 0
    }
    axis: 2
  }
}
layer {
  name: "cross_entropy_loss"
  type: "SoftmaxWithLoss"
  bottom: "predict"
  bottom: "target_sentence"
  top: "cross_entropy_loss"
  loss_weight: 20
  loss_param {
    ignore_label: -1
  }
  softmax_param {
    axis: 2
  }
}
layer {
  name: "accuracy"
  type: "Accuracy"
  bottom: "predict"
  bottom: "target_sentence"
  top: "accuracy"
  include { phase: TEST }
  accuracy_param {
    axis: 2
    ignore_label: -1
  }
}

While training, feed names as input and move names one character left as target output, i.e. input S-H-A-N-D-R-I-S, target H-A-N-D-R-I-S-‘\n’. The network will learn to predict what the next character will most probably be according to the context above. And if we direct the network’s predict output back to the input, the network will loops with its prediction and generate a list of names.

The result is really amazing. The sheet below shows two columns of names, one is generated by the LSTM network, and another is given by Blizzard staffs. Could you recognize which column is generated by the machine?

ClathielClarindrela
DorianaDahlia
ElanndiaElessaria
KulanaiKynreith
LanandrisLaria
Shyn'telShauana
TylariaTarindrella

C to HDL

Found a simple & generic way to translate programs into RTL. This used to be one of the hardest problems in my thought several years ago.

I came to this solution independently in April 2016, only known Xilinx/Altera have solved it recently. Unfortunately, many people have found and implemented this decades ago.

If you are looking for ideas about it, this example may help: c2hdl.

WoW Dictionary

Did some study about NLP.

We are planning to build a patch note bot, to publish patch changes to the forum automatically. I am working on the tooltips differences comparing algorithm.

Compare differences between strings are just simple with dynamic programming. If you don’t know how to do this just read this slide. That algorithm works on the granularity of alphabets. To scale the granularity to words, we need to do word split before we start this algorithm.

Splitting Latin words is easy. It won’t be more than 5 lines of code.

However, it is hard to split Chinese words. Billions of Chinese Ph.D. students graduate each year, by spamming all kinds of word segmentation methods. We need a dictionary before doing word segmentation. A dictionary for WoW.

I extracted all strings from DBC and filtered out zhCN corpus. Then implemented this algorithm.

The dictionary I got is ok for me. There are still garbage words in it, but rare. Some words are filtered out because they are rarely used in tooltips. E.g. “灰谷“(Ashenvale, I’m really unhappy about this :X) UPD 2016/4/13: Ashenvale is back now.

I would make my dictionary extractor open when the bot is ready. If you are looking for the dictionary too, you could get it from my github.

Hello World!

I am trying to migrate from Wordpress to Jekyll.

The old site could be found via this link if the cost is still under budget (the availability is pretty low, maybe ~10%).

The app engine provider significantly increased the cost, thus I have to quit, as a response.

Hello!