2025-04-06
@Shohei Yamagiwa
Created semantic analyzer to be able to make tokens of atomic, string and type tokens semantic
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Created semantic token based from Token.java
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Moved some tokens from Tokens.java to Constants.java, Functions.java and Types.java
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Moved comment tokens from Tokens.java to Comments.java
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Separate tokens into Tokens.java, Keywords.java and Operators.java
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Added package-info.java
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Changed method signatures
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Optimized some codes in Tokenizer#execute
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Optimized some codes in Tokenizer#splitBySpace
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Changed the way to split tokens by symbols and fixed known bugs.
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Removed some regex constants.
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Use String#length instead of String#codePointCount
Shohei Yamagiwa committed on 6 Apr
@Shohei Yamagiwa
Updated test declarations.
Shohei Yamagiwa committed on 6 Apr
2025-04-05
@Shohei Yamagiwa
Tokenizer#splitBySymbol is still buggy.
Shohei Yamagiwa committed on 5 Apr
@Shohei Yamagiwa
Added test case for Tokenizer#splitBySpace and Tokenizer#splitBySymbol
Shohei Yamagiwa committed on 5 Apr
2025-04-04
@Shohei Yamagiwa
Added test case for Tokenizer#splitBySymbol
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Changed access modifiers to be able to access methods in Tokenizer.java from TokenizerTest.java
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Added test case for Tokenizer#splitBySymbol.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Implemented Tokenizer#splitBySymbols to be able to tokenize by more symbols.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Implemented equals() to test in JUnit 5.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Implemented new constructor and fixed toString() and JavaDoc comments.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Added Position#move method.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Renamed some fields and methods in Token.java.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Made simple test-case for tokenizer.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Implemented model-file tokenizer that has abilities to tokenize comments and splits token by spaces.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Made skeleton of a model-file validator.
Shohei Yamagiwa committed on 4 Apr
@Shohei Yamagiwa
Implemented LSP4J to be able to launch basic language server.
Shohei Yamagiwa committed on 4 Apr