Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error Couldn't find <s> in first frame (ES language model) #48

Open
edwardffs opened this issue Feb 26, 2018 · 8 comments
Open

Error Couldn't find <s> in first frame (ES language model) #48

edwardffs opened this issue Feb 26, 2018 · 8 comments

Comments

@edwardffs
Copy link

I'm currently being flooded with this error in the logcat. The app still runs, but it recognizes the words very poorly. What could I be doing wrong?

@nshmyrev
Copy link
Contributor

You need to provide a bit more details on the code you use to get help.

@edwardffs
Copy link
Author

Thank you for the fast reply! Here's what I've done:

I downloaded these files to work with Spanish
https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/Spanish/

Reduced the Dictionary to only these words

anomalía a n o m a l i a
atrás a t r a s
listo l i s t o
listo(2) l i s t a
listo(3) l i s t a s
listo(4) l i s t o s
pausa p a u s a
pausa(2) p a u s a r
pausa(3) p a u t a s
repetir rr e p e t i r
repetir(2) rr e p e t i d o
repetir(3) rr e p e t i r s e
repetir(4) rr e p e t i r

And made my activity like this:

public class MainActivity extends AppCompatActivity implements RecognitionListener {

    private static final int PERMISSIONS_REQUEST_RECORD_AUDIO = 1;
    private static final String SEARCH = "helda";

    private SpeechRecognizer recognizer;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        int permissionCheck = ContextCompat.checkSelfPermission(getApplicationContext(), Manifest.permission.RECORD_AUDIO);
        if (permissionCheck != PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.RECORD_AUDIO}, PERMISSIONS_REQUEST_RECORD_AUDIO);
            return;
        }

        new SetupTask(this).execute();

    }

    private static class SetupTask extends AsyncTask<Void, Void, Exception> {
        WeakReference<MainActivity> activityReference;

        SetupTask(MainActivity activity) {
            this.activityReference = new WeakReference<>(activity);
        }

        @Override
        protected Exception doInBackground(Void... params) {
            try {
                Assets assets = new Assets(activityReference.get());
                File assetDir = assets.syncAssets();
                activityReference.get().setupRecognizer(assetDir);
            } catch (IOException e) {
                return e;
            }
            return null;
        }

        @Override
        protected void onPostExecute(Exception result) {
            if (result != null) {
                Log.d("STT LOG", "FAIL: " + result);
            } else {
                activityReference.get().switchSearch(SEARCH);
                Log.d("STT LOG", "POST EXECUTE SUCCESS");
            }
        }
    }

    @Override
    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);

        if (requestCode == PERMISSIONS_REQUEST_RECORD_AUDIO) {
            if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
                new SetupTask(this).execute();
            } else {
                finish();
            }
        }
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
        if (recognizer != null) {
            recognizer.cancel();
            recognizer.shutdown();
        }
    }

    @Override
    public void onPartialResult(Hypothesis hypothesis) {
        if (hypothesis != null) {
            Log.d("STTLOGGOT onPartial ", hypothesis.getHypstr());
        } else {
            Log.d("STTLOG onPartialResult", "HYPOTHESIS IS NULL");
        }
        switchSearch(SEARCH);
    }

    @Override
    public void onResult(Hypothesis hypothesis) {
        if (hypothesis != null) {
            Log.d("STTLOGGOT onResult ", hypothesis.getHypstr());
        } else {
            Log.d("STTLOG onResult", "HYPOTHESIS IS NULL");
        }
    }

    @Override
    public void onBeginningOfSpeech() {
        Log.d("STTLOG", "BEGINNING OF SPEECH");
    }

    @Override
    public void onEndOfSpeech() {
        Log.d("STTLOG", "END OF SPEECH");
        switchSearch(SEARCH);
    }

    private void switchSearch(String searchName) {
        recognizer.stop();
        if (searchName.equals(SEARCH)) {
            recognizer.startListening(searchName);
        } else {
            recognizer.startListening(searchName, 10);
        }
    }

    private void setupRecognizer(File assetsDir) throws IOException {

        recognizer = SpeechRecognizerSetup.defaultSetup()
                .setFloat("-vad_threshold", 2.0)
                .setAcousticModel(new File(assetsDir, "es-ptm"))
                .setDictionary(new File(assetsDir, "es.dict"))
                .getRecognizer();
        recognizer.addListener(this);
        File languageModel = new File(assetsDir, "es_model.lm");
        recognizer.addNgramSearch(SEARCH, languageModel);

        switchSearch(SEARCH);
    }

    @Override
    public void onError(Exception e) {
        Log.d("STTLOG", e.toString());
    }

    @Override
    public void onTimeout() {
        switchSearch(SEARCH);
    }
}

@nshmyrev
Copy link
Contributor

You need to provide es_model.lm

@edwardffs
Copy link
Author

As I said, I downloaded it from the link I gave:
https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/Spanish/
I didn't alter it at all

@nshmyrev
Copy link
Contributor

There is no file es_model.lm at that link.

@edwardffs
Copy link
Author

es_model.zip
It is the second one, but I attached it anyway

@nshmyrev
Copy link
Contributor

You need to provide logcat output and raw audio files.

@anuragsoftware
Copy link

instead of recognizer.startListening(searchName, 10);
you should use
recognizer.startListening(searchName, 10000); // i.e. 10 second sound recording buffer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants