Reusing the symmetric key in a hybrid cryptosystem may or may not be safe depending on the cryptosystem. To give a simple example, suppose a system was designed to use a single-use symmetric encryption key with a cipher in CTR mode, and it is now extended in such a way that the symmetric key is now reused for several messages but still shared with the same party. (If the protocol changes to share the symmetric key with a different party each time, that obviously affects security.) If the cryptosystem uses a random initial counter value, the same key could be used more than once without affecting security. If the cryptosystem uses the initial counter value 0, that's perfectly fine with a single-use key but catastrophic if the key is reused.
Having data which is encrypted with a long-term key which is unlocked at the beginning of a session is perfectly standard. Full-disk encryption does this, for example — typically the same key is used to encrypt all the successive versions of all the blocks of the encrypted device.
The basic idea of storing a symmetric key in a smartcard and having the card deliver it to the computer (a phone in your case) when the user enters a PIN is sound. The computer should keep the key only in RAM, and your application should wipe the key after a delay. How long a delay is a compromise between UX and security and depends on the nature of the data; you have to balance user convenience with the risk of the device being stolen.
There may or may not be value in encrypting each file with a different key. Having the card do the whole encryption and decryption is probably out of the question for both performance and UX reasons (the card would have to be present during all file accesses). But having the card release only one per-file key at a time may or may not make sense. You do need per-file keys if there is ever a reason why access to different files should be subject to different rules (e.g. you might want different timeouts on different files' keys). Per-file keys also offer better protection against a limited-time compromise (e.g. a rogue app that can extract the key from your app's memory but not inject code into your app). You can have per-file keys in two ways: either there's a single master key which the card releases upon request, and the computer does the key derivations; or the master key never leaves the card and the computer asks the card to derive each file's key when that file is accessed.
Whatever algorithm you pick, make sure to use it properly, in particular with a random IV for each file (stored at the beginning of the file). Note that if a file changes, the new version of the file must use a new random IV if it's encrypted with the same key.
You should use authenticated encryption (e.g. GCM), to prevent active attacks where a rogue app on the computer tries to decrypt bad data. Note that this means that a file must be decrypted (+authenticated) as a whole; you may need to split large files. Note also that authenticating a file only proves that it's some valid version of the file; it could be an old version, or it could be valid content of a different file unless you authenticate the relationship between a file's name and its content.
As a final note, modern major mobile platforms include protected environments that can store encrypted files with some protection against third-party applications and some form of user authentication (PIN or fingerprint) required for access (keychain on iPhone/iPad, keymaster on high-end Android phones and tablets). This satisfies the requirement of two-factor authentication. While it doesn't offer as a good a protection against attacks against the key at rest, it offers a better protection over what is a significant threat here, which is a rogue app on the device exploiting an OS bug to extract the key while it's in use. With the smartcard approach, the key is in your app's memory; with a keychain/keymaster approach, the key remains inside a protected environment, although the decrypted data would be visible to the rogue app anyway.
